[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh]
Translate memory configuration docs into Chinese.
URL: https://github.com/apache/flink/pull/11401#discussion_r392800869
##
File path: docs/ops/memory/mem_tuning.zh.md
##
@@ -22,66 +22,55 @@ specific language governing permissions and limitations
under the License.
-->
-In addition to the [main memory setup guide](mem_setup.html), this section
explains how to setup memory of task executors
-depending on the use case and which options are important in which case.
+本文在的基本的[配置指南](mem_setup.html)的基础上，介绍如何根据具体的使用场景调整 TaskExecutor
的内存配置，以及在不同使用场景下分别需要重点关注哪些配置参数。
* toc
{:toc}
-## Configure memory for standalone deployment
+## 独立部署模式（Standalone Deployment）下的内存配置
-It is recommended to configure [total Flink memory](mem_setup.html#配置总内存)
-([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size))
or its [components](mem_detail.html)
-for [standalone deployment](../deployment/cluster_setup.html) where you want
to declare how much memory is given to Flink itself.
-Additionally, you can adjust *JVM metaspace* if it causes
[problems](mem_trouble.html#outofmemoryerror-metaspace).
+[独立部署模式](../deployment/cluster_setup.html)，我们通常更关注 Flink 应用本身使用的内存大小。
+建议配置 [Flink
总内存](mem_setup.html#配置总内存)（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)）或者它的[组成部分](mem_detail.html)。
+此外，如果出现 [Metaspace 不足的问题](mem_trouble.html#outofmemoryerror-metaspace)，可以调整
*JVM Metaspace* 的大小。
-The *total Process memory* is not relevant because *JVM overhead* is not
controlled by Flink or deployment environment,
-only physical resources of the executing machine matter in this case.
+这种情况下通常无需配置*进程总内存*，因为不管是 Flink 还是部署环境都不会对 *JVM 开销* 进行限制，它只与机器的物理资源相关。
-## Configure memory for containers
+## 容器（Container）的内存配置
-It is recommended to configure [total process memory](mem_setup.html#配置总内存)
-([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size))
for the containerized deployments
-([Kubernetes](../deployment/kubernetes.html),
[Yarn](../deployment/yarn_setup.html) or [Mesos](../deployment/mesos.html)).
-It declares how much memory in total should be assigned to the Flink *JVM
process* and corresponds to the size of the requested container.
+在容器化部署模式（Containerized
Deployment）下（[Kubernetes](../deployment/kubernetes.html)、[Yarn](../deployment/yarn_setup.html)
或
[Mesos](../deployment/mesos.html)），建议配置[进程总内存](mem_setup.html#配置总内存)（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)）。
+该配置参数用于指定分配给 Flink *JVM 进程*的总内存，也就是需要申请的容器大小。
-Note If you configure the *total Flink
memory* Flink will implicitly add JVM memory components
-to derive the *total process memory* and request a container with the memory
of that derived size,
-see also [detailed Memory Model](mem_detail.html).
+提示 如果配置了 *Flink 总内存*，Flink 会自动加上 JVM
相关的内存部分，根据推算出的*进程总内存*大小申请容器。
+请参考[内存模型详解](mem_detail.html)。
- Warning: If Flink or user code allocates unmanaged off-heap
(native) memory beyond the container size
- the job can fail because the deployment environment can kill the offending
containers.
+ 注意： 如果 Flink
或者用户代码分配超过容器大小的非托管的堆外（本地）内存，部署环境可能会杀掉超用内存的容器，造成作业执行失败。
-See also description of [container memory
exceeded](mem_trouble.html#container-memory-exceeded) failure.
+请参考[容器内存超用](mem_trouble.html#container-memory-exceeded)中的相关描述。
-## Configure memory for state backends
+## State Backend 的内存配置
-When deploying a Flink streaming application, the type of [state
backend](../state/state_backends.html) used
-will dictate the optimal memory configurations of your cluster.
+在部署 Flink 流处理应用时，可以根据 [State Backend](../state/state_backends.html)
的类型对集群的配置进行优化。
-### Heap state backend
+### Heap State Backend
-When running a stateless job or using a heap state backend
([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
-or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)), set
[managed memory](mem_setup.html#托管内存) to zero.
-This will ensure that the maximum amount of memory is allocated for user code
on the JVM.
+执行无状态作业或者使用 Heap State
Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
+或
[FsStateBackend](../state/state_backends.html#the-fsstatebackend)）时，建议将[托管内存](mem_setup.html#托管内存)设置为
0。
Review comment:
Broken link. Should be #fsstatebackend
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh]
Translate memory configuration docs into Chinese.
URL: https://github.com/apache/flink/pull/11401#discussion_r392801162
##
File path: docs/ops/memory/mem_tuning.zh.md
##
@@ -22,66 +22,55 @@ specific language governing permissions and limitations
under the License.
-->
-In addition to the [main memory setup guide](mem_setup.html), this section
explains how to setup memory of task executors
-depending on the use case and which options are important in which case.
+本文在的基本的[配置指南](mem_setup.html)的基础上，介绍如何根据具体的使用场景调整 TaskExecutor
的内存配置，以及在不同使用场景下分别需要重点关注哪些配置参数。
* toc
{:toc}
-## Configure memory for standalone deployment
+## 独立部署模式（Standalone Deployment）下的内存配置
-It is recommended to configure [total Flink memory](mem_setup.html#配置总内存)
-([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size))
or its [components](mem_detail.html)
-for [standalone deployment](../deployment/cluster_setup.html) where you want
to declare how much memory is given to Flink itself.
-Additionally, you can adjust *JVM metaspace* if it causes
[problems](mem_trouble.html#outofmemoryerror-metaspace).
+[独立部署模式](../deployment/cluster_setup.html)，我们通常更关注 Flink 应用本身使用的内存大小。
+建议配置 [Flink
总内存](mem_setup.html#配置总内存)（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)）或者它的[组成部分](mem_detail.html)。
+此外，如果出现 [Metaspace 不足的问题](mem_trouble.html#outofmemoryerror-metaspace)，可以调整
*JVM Metaspace* 的大小。
-The *total Process memory* is not relevant because *JVM overhead* is not
controlled by Flink or deployment environment,
-only physical resources of the executing machine matter in this case.
+这种情况下通常无需配置*进程总内存*，因为不管是 Flink 还是部署环境都不会对 *JVM 开销* 进行限制，它只与机器的物理资源相关。
-## Configure memory for containers
+## 容器（Container）的内存配置
-It is recommended to configure [total process memory](mem_setup.html#配置总内存)
-([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size))
for the containerized deployments
-([Kubernetes](../deployment/kubernetes.html),
[Yarn](../deployment/yarn_setup.html) or [Mesos](../deployment/mesos.html)).
-It declares how much memory in total should be assigned to the Flink *JVM
process* and corresponds to the size of the requested container.
+在容器化部署模式（Containerized
Deployment）下（[Kubernetes](../deployment/kubernetes.html)、[Yarn](../deployment/yarn_setup.html)
或
[Mesos](../deployment/mesos.html)），建议配置[进程总内存](mem_setup.html#配置总内存)（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)）。
+该配置参数用于指定分配给 Flink *JVM 进程*的总内存，也就是需要申请的容器大小。
-Note If you configure the *total Flink
memory* Flink will implicitly add JVM memory components
-to derive the *total process memory* and request a container with the memory
of that derived size,
-see also [detailed Memory Model](mem_detail.html).
+提示 如果配置了 *Flink 总内存*，Flink 会自动加上 JVM
相关的内存部分，根据推算出的*进程总内存*大小申请容器。
+请参考[内存模型详解](mem_detail.html)。
- Warning: If Flink or user code allocates unmanaged off-heap
(native) memory beyond the container size
- the job can fail because the deployment environment can kill the offending
containers.
+ 注意： 如果 Flink
或者用户代码分配超过容器大小的非托管的堆外（本地）内存，部署环境可能会杀掉超用内存的容器，造成作业执行失败。
-See also description of [container memory
exceeded](mem_trouble.html#container-memory-exceeded) failure.
+请参考[容器内存超用](mem_trouble.html#container-memory-exceeded)中的相关描述。
-## Configure memory for state backends
+## State Backend 的内存配置
-When deploying a Flink streaming application, the type of [state
backend](../state/state_backends.html) used
-will dictate the optimal memory configurations of your cluster.
+在部署 Flink 流处理应用时，可以根据 [State Backend](../state/state_backends.html)
的类型对集群的配置进行优化。
-### Heap state backend
+### Heap State Backend
-When running a stateless job or using a heap state backend
([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
-or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)), set
[managed memory](mem_setup.html#托管内存) to zero.
-This will ensure that the maximum amount of memory is allocated for user code
on the JVM.
+执行无状态作业或者使用 Heap State
Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
+或
[FsStateBackend](../state/state_backends.html#the-fsstatebackend)）时，建议将[托管内存](mem_setup.html#托管内存)设置为
0。
+这样能够最大化分配给 JVM 上用户代码的内存。
-### RocksDB state backend
+### RocksDB State Backend
-The
[RocksDBStateBackend](../state/state_backends.html#the-rocksdbstatebackend)
uses native memory. By default,
-RocksDB is setup to limit native memory allocation to the size of the [managed
memory](mem_setup.html#托管内存).
-Therefore, it is important to reserve enough *managed memory* for your state
use case. If you disable the default RocksDB memory control,
-task executors can be killed in containerized deployments if RocksDB allocates
memory above the limit of the requested container size
-
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392805878 ## File path: docs/ops/memory/mem_migration.zh.md ## @@ -121,116 +117,105 @@ The following options are deprecated but if they are still used they will be int -Although, the network memory configuration has not changed too much it is recommended to verify its configuration. -It can change if other memory components have new sizes, e.g. the total memory which the network can be a fraction of. -See also [new detailed memory model](mem_detail.html). +尽管网络内存的配置参数没有发生太多变化，我们仍建议您检查其配置结果。 +网络内存的大小可能会受到其他内存部分大小变化的影响，例如总内存变化时，根据占比计算出的网络内存也可能发生变化。 +请参考[内存模型详解](mem_detail.html)。 -The container cut-off configuration options, [`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) -and [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min), have no effect for task manager processes anymore -but they still have the same semantics for the job manager process. See also [how to migrate container cut-off](#container-cut-off-memory). +容器切除（Cut-Off）内存相关的配置参数（[`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) +和 [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min)）将不再对 TaskExecutor 进程生效。 +对于 JobManager 进程，它们仍具有与此前相同的语义。 +请参考[如何升级容器切除内存](#容器切除cut-off内存)。 -## Total Memory (Previously Heap Memory) +## 总内存（原堆内存） -The previous options which were responsible for the total memory used by Flink are `taskmanager.heap.size` or `taskmanager.heap.mb`. -Despite their naming, they included not only JVM heap but also other off-heap memory components. The options have been deprecated. +在原本的内存配置方法中，用于指定用于 Flink 的总内存的配置参数是 `taskmanager.heap.size` 或 `taskmanager.heap.mb`。 +尽管这两个参数以“堆（Heap）”命名，实际上它们指定的内存既包含了 JVM 堆内存，也包含了其他堆外内存部分。 +这两个配置参数目前已被弃用。 -The Mesos integration also had a separate option with the same semantics: `mesos.resourcemanager.tasks.mem` which has also been removed. +Flink 在 Mesos 上还有另一个具有同样语义的配置参数 `mesos.resourcemanager.tasks.mem`，目前也已经被弃用。 -If the mentioned legacy options are used without specifying the corresponding new options, -they will be directly translated into the following new options: -* Total Flink memory ([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) for standalone deployments -* Total process memory ([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)) for containerized deployments (Yarn or Mesos) +如果配置了上述弃用的参数，同时又没有配置与之对应的新配置参数，那它们将按如下规则对应到新的配置参数。 +* 独立部署模式（Standalone Deployment）下：Flink 总内存（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)） +* 容器化部署模式（Containerized Deployement）下（Yarn、Mesos）：进程总内存（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)） -It is also recommended to use these new options instead of the legacy ones as they might be completely removed in the following releases. +建议您尽早使用新的配置参数取代启用的配置参数，它们在今后的版本中可能会被彻底移除。 -See also [how to configure total memory now](mem_setup.html#配置总内存). +请参考[如何配置总内存](mem_setup.html#配置总内存). -## JVM Heap Memory +## JVM 堆内存 -JVM heap memory previously consisted of the managed memory (if configured to be on-heap) and the rest -which included any other usages of heap memory. This rest was always implicitly derived as the remaining part of the total memory, -see also [how to migrate managed memory](#managed-memory). +此前，JVM 堆空间由托管内存（仅在配置为堆上时）及 Flink 用到的所有其他堆内存组成。 +这里的其他堆内存是由总内存减去所有其他非堆内存得到的。 +请参考[如何升级托管内存](#托管内存)。 -Now, if only *total Flink memory* or *total process memory* is configured, then the JVM heap is also derived as the rest of -what is left after subtracting all other components from the total memory, see also [how to configure total memory](mem_setup.html#配置总内存). +现在，如果仅配置了*Flink总内存*或*进程总内存*，JVM 的堆空间依然是根据总内存减去所有其他非堆内存得到的。 +请参考[如何配置总内存](mem_setup.html#配置总内存)。 -Additionally, you can now have more direct control over the JVM heap assigned to the operator tasks -([`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)), -see also [Task (Operator) Heap Memory](mem_setup.html#任务算子堆内存). -The JVM heap memory is also used by the heap state backends ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) -or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)) if it is chosen for streaming jobs. +此外，你现在可以更直接地控制用于任务和算子的 JVM 的堆内存（[`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)），详见[任务堆内存](mem_setup.html#任务算子堆内存)。 +如果流处理作业选择使用 Heap State Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) Review comment: Broken link. This is an automated message from the Apache Git Service. T
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392803707 ## File path: docs/ops/memory/mem_trouble.zh.md ## @@ -27,48 +27,43 @@ under the License. ## IllegalConfigurationException -If you see an *IllegalConfigurationException* thrown from *TaskExecutorProcessUtils*, it usually indicates -that there is either an invalid configuration value (e.g. negative memory size, fraction that is greater than 1, etc.) -or configuration conflicts. Check the documentation chapters related to the [memory components](mem_detail.html) -mentioned in the exception message. +如果遇到从 *TaskExecutorProcessUtils* 抛出的 *IllegalConfigurationException* 异常，这通常说明您的配置参数中存在无效值（例如内存大小为负数、占比大于 1 等）或者配置冲突。 +请根据异常信息，确认[内存模型详解](mem_detail.html)中与出错的内存部分对应章节的内容。 ## OutOfMemoryError: Java heap space -The exception usually indicates that the JVM heap is too small. You can try to increase the JVM heap size -by increasing [total memory](mem_setup.html#配置总内存) or [task heap memory](mem_setup.html#任务算子堆内存). +该异常说明 JVM 的堆空间过小。 +可以通过增大[总内存](mem_setup.html#配置总内存)或[任务堆内存](mem_setup.html#任务算子堆内存)的方法来增大 JVM 堆空间。 -Note You can also increase the [framework heap memory](mem_detail.html#框架内存) but this option -is advanced and should only be changed if you are sure that the Flink framework itself needs more memory. +提示 也可以增大[框架堆内存](mem_detail.html#框架内存)。这是一个进阶配置，只有在确认是 Flink 框架自身需要更多内存时才应该去调整。 ## OutOfMemoryError: Direct buffer memory -The exception usually indicates that the JVM *direct memory* limit is too small or that there is a *direct memory leak*. -Check whether user code or other external dependencies use the JVM *direct memory* and that it is properly accounted for. -You can try to increase its limit by adjusting [direct off-heap memory](mem_detail.html). -See also [how to configure off-heap memory](mem_setup.html#配置堆外内存直接内存或本地内存)) and -the [JVM arguments](mem_detail.html#jvm-参数) which Flink sets. +该异常通常说明 JVM 的*直接内存*限制过小，或者存在*直接内存泄漏（Direct Memory Leak）*。 +请确认用户代码及外部依赖中是否使用了 JVM *直接内存*，以及如果使用了直接内存，是否配置了足够的内存空间。 +可以通过调整[堆外内存](mem_detail.html)来增大直接内存限制。 +请同时参考[如何配置堆外内存](mem_setup.html#配置堆外内存直接内存或本地内存))以及 Flink 设置的 [JVM 参数](mem_detail.html#jvm-参数)。 ## OutOfMemoryError: Metaspace -The exception usually indicates that [JVM metaspace limit](mem_detail.html#jvm-参数) is configured too small. -You can try to increase the [JVM metaspace option](../config.html#taskmanager-memory-jvm-metaspace-size). +该异常说明 [JVM Metaspace 限制](mem_detail.html#jvm-参数)过小。 +可以尝试调整 [JVM Metaspace 参数](../config.html#taskmanager-memory-jvm-metaspace-size)。 ## IOException: Insufficient number of network buffers -The exception usually indicates that the size of the configured [network memory](mem_detail.html) -is not big enough. You can try to increase the *network memory* by adjusting the following options: +该异常通常说明[网络内存](mem_detail.html)过小。 +可以通过调整以下配置参数增大*网络内存*： * [`taskmanager.memory.network.min`](../config.html#taskmanager-memory-network-min) * [`taskmanager.memory.network.max`](../config.html#taskmanager-memory-network-max) * [`taskmanager.memory.network.fraction`](../config.html#taskmanager-memory-network-fraction) -## Container Memory Exceeded +## 容器（Container）内存超用 -If a task executor container tries to allocate memory beyond its requested size (Yarn, Mesos or Kubernetes), -this usually indicates that Flink has not reserved enough native memory. You can observe this either by using an external -monitoring system or from the error messages when a container gets killed by the deployment environment. +如果 TaskExecutor 容器尝试分配超过其申请大小的内存（Yarn、Mesos 或 Kubernetes），者通常说明 Flink 没有预留出足够的本地内存。 +可以通过外部监控系统或者容器被部署环境杀掉时的错误信息判断是否存在容器内存超用。 -If [RocksDBStateBackend](../state/state_backends.html#the-rocksdbstatebackend) is used and the memory controlling is disabled, -you can try to increase the [managed memory](mem_setup.html#托管内存). +如果使用了 [RocksDBStateBackend](../state/state_backends.html#the-rocksdbstatebackend) 且没有开启内存控制，也可以尝试增大[托管内存](mem_setup.html#托管内存)。 -Alternatively, you can increase the [JVM overhead](mem_detail.html). -See also [how to configure memory for containers](mem_tuning.html#容器container的内存配置). +此外，还可以尝试增大 [JVM 开销](mem_detail.html)。 Review comment: #受限的等比内存部分 could be better. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392801817 ## File path: docs/ops/memory/mem_trouble.zh.md ## @@ -27,48 +27,43 @@ under the License. ## IllegalConfigurationException -If you see an *IllegalConfigurationException* thrown from *TaskExecutorProcessUtils*, it usually indicates -that there is either an invalid configuration value (e.g. negative memory size, fraction that is greater than 1, etc.) -or configuration conflicts. Check the documentation chapters related to the [memory components](mem_detail.html) -mentioned in the exception message. +如果遇到从 *TaskExecutorProcessUtils* 抛出的 *IllegalConfigurationException* 异常，这通常说明您的配置参数中存在无效值（例如内存大小为负数、占比大于 1 等）或者配置冲突。 +请根据异常信息，确认[内存模型详解](mem_detail.html)中与出错的内存部分对应章节的内容。 ## OutOfMemoryError: Java heap space -The exception usually indicates that the JVM heap is too small. You can try to increase the JVM heap size -by increasing [total memory](mem_setup.html#配置总内存) or [task heap memory](mem_setup.html#任务算子堆内存). +该异常说明 JVM 的堆空间过小。 +可以通过增大[总内存](mem_setup.html#配置总内存)或[任务堆内存](mem_setup.html#任务算子堆内存)的方法来增大 JVM 堆空间。 -Note You can also increase the [framework heap memory](mem_detail.html#框架内存) but this option -is advanced and should only be changed if you are sure that the Flink framework itself needs more memory. +提示 也可以增大[框架堆内存](mem_detail.html#框架内存)。这是一个进阶配置，只有在确认是 Flink 框架自身需要更多内存时才应该去调整。 ## OutOfMemoryError: Direct buffer memory -The exception usually indicates that the JVM *direct memory* limit is too small or that there is a *direct memory leak*. -Check whether user code or other external dependencies use the JVM *direct memory* and that it is properly accounted for. -You can try to increase its limit by adjusting [direct off-heap memory](mem_detail.html). -See also [how to configure off-heap memory](mem_setup.html#配置堆外内存直接内存或本地内存)) and -the [JVM arguments](mem_detail.html#jvm-参数) which Flink sets. +该异常通常说明 JVM 的*直接内存*限制过小，或者存在*直接内存泄漏（Direct Memory Leak）*。 +请确认用户代码及外部依赖中是否使用了 JVM *直接内存*，以及如果使用了直接内存，是否配置了足够的内存空间。 +可以通过调整[堆外内存](mem_detail.html)来增大直接内存限制。 +请同时参考[如何配置堆外内存](mem_setup.html#配置堆外内存直接内存或本地内存))以及 Flink 设置的 [JVM 参数](mem_detail.html#jvm-参数)。 ## OutOfMemoryError: Metaspace -The exception usually indicates that [JVM metaspace limit](mem_detail.html#jvm-参数) is configured too small. -You can try to increase the [JVM metaspace option](../config.html#taskmanager-memory-jvm-metaspace-size). +该异常说明 [JVM Metaspace 限制](mem_detail.html#jvm-参数)过小。 +可以尝试调整 [JVM Metaspace 参数](../config.html#taskmanager-memory-jvm-metaspace-size)。 ## IOException: Insufficient number of network buffers -The exception usually indicates that the size of the configured [network memory](mem_detail.html) -is not big enough. You can try to increase the *network memory* by adjusting the following options: +该异常通常说明[网络内存](mem_detail.html)过小。 +可以通过调整以下配置参数增大*网络内存*： * [`taskmanager.memory.network.min`](../config.html#taskmanager-memory-network-min) * [`taskmanager.memory.network.max`](../config.html#taskmanager-memory-network-max) * [`taskmanager.memory.network.fraction`](../config.html#taskmanager-memory-network-fraction) -## Container Memory Exceeded +## 容器（Container）内存超用 -If a task executor container tries to allocate memory beyond its requested size (Yarn, Mesos or Kubernetes), -this usually indicates that Flink has not reserved enough native memory. You can observe this either by using an external -monitoring system or from the error messages when a container gets killed by the deployment environment. +如果 TaskExecutor 容器尝试分配超过其申请大小的内存（Yarn、Mesos 或 Kubernetes），者通常说明 Flink 没有预留出足够的本地内存。 Review comment: Typo: 者 -> 这 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392803428 ## File path: docs/ops/memory/mem_trouble.zh.md ## @@ -27,48 +27,43 @@ under the License. ## IllegalConfigurationException -If you see an *IllegalConfigurationException* thrown from *TaskExecutorProcessUtils*, it usually indicates -that there is either an invalid configuration value (e.g. negative memory size, fraction that is greater than 1, etc.) -or configuration conflicts. Check the documentation chapters related to the [memory components](mem_detail.html) -mentioned in the exception message. +如果遇到从 *TaskExecutorProcessUtils* 抛出的 *IllegalConfigurationException* 异常，这通常说明您的配置参数中存在无效值（例如内存大小为负数、占比大于 1 等）或者配置冲突。 +请根据异常信息，确认[内存模型详解](mem_detail.html)中与出错的内存部分对应章节的内容。 ## OutOfMemoryError: Java heap space -The exception usually indicates that the JVM heap is too small. You can try to increase the JVM heap size -by increasing [total memory](mem_setup.html#配置总内存) or [task heap memory](mem_setup.html#任务算子堆内存). +该异常说明 JVM 的堆空间过小。 +可以通过增大[总内存](mem_setup.html#配置总内存)或[任务堆内存](mem_setup.html#任务算子堆内存)的方法来增大 JVM 堆空间。 -Note You can also increase the [framework heap memory](mem_detail.html#框架内存) but this option -is advanced and should only be changed if you are sure that the Flink framework itself needs more memory. +提示 也可以增大[框架堆内存](mem_detail.html#框架内存)。这是一个进阶配置，只有在确认是 Flink 框架自身需要更多内存时才应该去调整。 ## OutOfMemoryError: Direct buffer memory -The exception usually indicates that the JVM *direct memory* limit is too small or that there is a *direct memory leak*. -Check whether user code or other external dependencies use the JVM *direct memory* and that it is properly accounted for. -You can try to increase its limit by adjusting [direct off-heap memory](mem_detail.html). -See also [how to configure off-heap memory](mem_setup.html#配置堆外内存直接内存或本地内存)) and -the [JVM arguments](mem_detail.html#jvm-参数) which Flink sets. +该异常通常说明 JVM 的*直接内存*限制过小，或者存在*直接内存泄漏（Direct Memory Leak）*。 +请确认用户代码及外部依赖中是否使用了 JVM *直接内存*，以及如果使用了直接内存，是否配置了足够的内存空间。 +可以通过调整[堆外内存](mem_detail.html)来增大直接内存限制。 +请同时参考[如何配置堆外内存](mem_setup.html#配置堆外内存直接内存或本地内存))以及 Flink 设置的 [JVM 参数](mem_detail.html#jvm-参数)。 ## OutOfMemoryError: Metaspace -The exception usually indicates that [JVM metaspace limit](mem_detail.html#jvm-参数) is configured too small. -You can try to increase the [JVM metaspace option](../config.html#taskmanager-memory-jvm-metaspace-size). +该异常说明 [JVM Metaspace 限制](mem_detail.html#jvm-参数)过小。 +可以尝试调整 [JVM Metaspace 参数](../config.html#taskmanager-memory-jvm-metaspace-size)。 ## IOException: Insufficient number of network buffers -The exception usually indicates that the size of the configured [network memory](mem_detail.html) -is not big enough. You can try to increase the *network memory* by adjusting the following options: +该异常通常说明[网络内存](mem_detail.html)过小。 +可以通过调整以下配置参数增大*网络内存*： * [`taskmanager.memory.network.min`](../config.html#taskmanager-memory-network-min) * [`taskmanager.memory.network.max`](../config.html#taskmanager-memory-network-max) * [`taskmanager.memory.network.fraction`](../config.html#taskmanager-memory-network-fraction) -## Container Memory Exceeded +## 容器（Container）内存超用 -If a task executor container tries to allocate memory beyond its requested size (Yarn, Mesos or Kubernetes), -this usually indicates that Flink has not reserved enough native memory. You can observe this either by using an external -monitoring system or from the error messages when a container gets killed by the deployment environment. +如果 TaskExecutor 容器尝试分配超过其申请大小的内存（Yarn、Mesos 或 Kubernetes），者通常说明 Flink 没有预留出足够的本地内存。 +可以通过外部监控系统或者容器被部署环境杀掉时的错误信息判断是否存在容器内存超用。 -If [RocksDBStateBackend](../state/state_backends.html#the-rocksdbstatebackend) is used and the memory controlling is disabled, -you can try to increase the [managed memory](mem_setup.html#托管内存). +如果使用了 [RocksDBStateBackend](../state/state_backends.html#the-rocksdbstatebackend) 且没有开启内存控制，也可以尝试增大[托管内存](mem_setup.html#托管内存)。 Review comment: Broken link. Should be #rocksdbstatebackend . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392811503 ## File path: docs/ops/memory/mem_migration.zh.md ## @@ -121,116 +117,105 @@ The following options are deprecated but if they are still used they will be int -Although, the network memory configuration has not changed too much it is recommended to verify its configuration. -It can change if other memory components have new sizes, e.g. the total memory which the network can be a fraction of. -See also [new detailed memory model](mem_detail.html). +尽管网络内存的配置参数没有发生太多变化，我们仍建议您检查其配置结果。 +网络内存的大小可能会受到其他内存部分大小变化的影响，例如总内存变化时，根据占比计算出的网络内存也可能发生变化。 +请参考[内存模型详解](mem_detail.html)。 -The container cut-off configuration options, [`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) -and [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min), have no effect for task manager processes anymore -but they still have the same semantics for the job manager process. See also [how to migrate container cut-off](#container-cut-off-memory). +容器切除（Cut-Off）内存相关的配置参数（[`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) +和 [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min)）将不再对 TaskExecutor 进程生效。 +对于 JobManager 进程，它们仍具有与此前相同的语义。 +请参考[如何升级容器切除内存](#容器切除cut-off内存)。 -## Total Memory (Previously Heap Memory) +## 总内存（原堆内存） -The previous options which were responsible for the total memory used by Flink are `taskmanager.heap.size` or `taskmanager.heap.mb`. -Despite their naming, they included not only JVM heap but also other off-heap memory components. The options have been deprecated. +在原本的内存配置方法中，用于指定用于 Flink 的总内存的配置参数是 `taskmanager.heap.size` 或 `taskmanager.heap.mb`。 +尽管这两个参数以“堆（Heap）”命名，实际上它们指定的内存既包含了 JVM 堆内存，也包含了其他堆外内存部分。 +这两个配置参数目前已被弃用。 -The Mesos integration also had a separate option with the same semantics: `mesos.resourcemanager.tasks.mem` which has also been removed. +Flink 在 Mesos 上还有另一个具有同样语义的配置参数 `mesos.resourcemanager.tasks.mem`，目前也已经被弃用。 -If the mentioned legacy options are used without specifying the corresponding new options, -they will be directly translated into the following new options: -* Total Flink memory ([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) for standalone deployments -* Total process memory ([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)) for containerized deployments (Yarn or Mesos) +如果配置了上述弃用的参数，同时又没有配置与之对应的新配置参数，那它们将按如下规则对应到新的配置参数。 +* 独立部署模式（Standalone Deployment）下：Flink 总内存（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)） +* 容器化部署模式（Containerized Deployement）下（Yarn、Mesos）：进程总内存（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)） -It is also recommended to use these new options instead of the legacy ones as they might be completely removed in the following releases. +建议您尽早使用新的配置参数取代启用的配置参数，它们在今后的版本中可能会被彻底移除。 -See also [how to configure total memory now](mem_setup.html#配置总内存). +请参考[如何配置总内存](mem_setup.html#配置总内存). -## JVM Heap Memory +## JVM 堆内存 -JVM heap memory previously consisted of the managed memory (if configured to be on-heap) and the rest -which included any other usages of heap memory. This rest was always implicitly derived as the remaining part of the total memory, -see also [how to migrate managed memory](#managed-memory). +此前，JVM 堆空间由托管内存（仅在配置为堆上时）及 Flink 用到的所有其他堆内存组成。 +这里的其他堆内存是由总内存减去所有其他非堆内存得到的。 +请参考[如何升级托管内存](#托管内存)。 -Now, if only *total Flink memory* or *total process memory* is configured, then the JVM heap is also derived as the rest of -what is left after subtracting all other components from the total memory, see also [how to configure total memory](mem_setup.html#配置总内存). +现在，如果仅配置了*Flink总内存*或*进程总内存*，JVM 的堆空间依然是根据总内存减去所有其他非堆内存得到的。 +请参考[如何配置总内存](mem_setup.html#配置总内存)。 -Additionally, you can now have more direct control over the JVM heap assigned to the operator tasks -([`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)), -see also [Task (Operator) Heap Memory](mem_setup.html#任务算子堆内存). -The JVM heap memory is also used by the heap state backends ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) -or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)) if it is chosen for streaming jobs. +此外，你现在可以更直接地控制用于任务和算子的 JVM 的堆内存（[`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)），详见[任务堆内存](mem_setup.html#任务算子堆内存)。 +如果流处理作业选择使用 Heap State Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) +或 [FsStateBackend](../state/state_backends.html#the-fsstatebackend)），那么它同样需要使用 JVM 堆内存。 -A part of the JVM heap is now always reserved for Flink framework -
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh]
Translate memory configuration docs into Chinese.
URL: https://github.com/apache/flink/pull/11401#discussion_r392800746
##
File path: docs/ops/memory/mem_tuning.zh.md
##
@@ -22,66 +22,55 @@ specific language governing permissions and limitations
under the License.
-->
-In addition to the [main memory setup guide](mem_setup.html), this section
explains how to setup memory of task executors
-depending on the use case and which options are important in which case.
+本文在的基本的[配置指南](mem_setup.html)的基础上，介绍如何根据具体的使用场景调整 TaskExecutor
的内存配置，以及在不同使用场景下分别需要重点关注哪些配置参数。
* toc
{:toc}
-## Configure memory for standalone deployment
+## 独立部署模式（Standalone Deployment）下的内存配置
-It is recommended to configure [total Flink memory](mem_setup.html#配置总内存)
-([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size))
or its [components](mem_detail.html)
-for [standalone deployment](../deployment/cluster_setup.html) where you want
to declare how much memory is given to Flink itself.
-Additionally, you can adjust *JVM metaspace* if it causes
[problems](mem_trouble.html#outofmemoryerror-metaspace).
+[独立部署模式](../deployment/cluster_setup.html)，我们通常更关注 Flink 应用本身使用的内存大小。
+建议配置 [Flink
总内存](mem_setup.html#配置总内存)（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)）或者它的[组成部分](mem_detail.html)。
+此外，如果出现 [Metaspace 不足的问题](mem_trouble.html#outofmemoryerror-metaspace)，可以调整
*JVM Metaspace* 的大小。
-The *total Process memory* is not relevant because *JVM overhead* is not
controlled by Flink or deployment environment,
-only physical resources of the executing machine matter in this case.
+这种情况下通常无需配置*进程总内存*，因为不管是 Flink 还是部署环境都不会对 *JVM 开销* 进行限制，它只与机器的物理资源相关。
-## Configure memory for containers
+## 容器（Container）的内存配置
-It is recommended to configure [total process memory](mem_setup.html#配置总内存)
-([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size))
for the containerized deployments
-([Kubernetes](../deployment/kubernetes.html),
[Yarn](../deployment/yarn_setup.html) or [Mesos](../deployment/mesos.html)).
-It declares how much memory in total should be assigned to the Flink *JVM
process* and corresponds to the size of the requested container.
+在容器化部署模式（Containerized
Deployment）下（[Kubernetes](../deployment/kubernetes.html)、[Yarn](../deployment/yarn_setup.html)
或
[Mesos](../deployment/mesos.html)），建议配置[进程总内存](mem_setup.html#配置总内存)（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)）。
+该配置参数用于指定分配给 Flink *JVM 进程*的总内存，也就是需要申请的容器大小。
-Note If you configure the *total Flink
memory* Flink will implicitly add JVM memory components
-to derive the *total process memory* and request a container with the memory
of that derived size,
-see also [detailed Memory Model](mem_detail.html).
+提示 如果配置了 *Flink 总内存*，Flink 会自动加上 JVM
相关的内存部分，根据推算出的*进程总内存*大小申请容器。
+请参考[内存模型详解](mem_detail.html)。
- Warning: If Flink or user code allocates unmanaged off-heap
(native) memory beyond the container size
- the job can fail because the deployment environment can kill the offending
containers.
+ 注意： 如果 Flink
或者用户代码分配超过容器大小的非托管的堆外（本地）内存，部署环境可能会杀掉超用内存的容器，造成作业执行失败。
-See also description of [container memory
exceeded](mem_trouble.html#container-memory-exceeded) failure.
+请参考[容器内存超用](mem_trouble.html#container-memory-exceeded)中的相关描述。
-## Configure memory for state backends
+## State Backend 的内存配置
-When deploying a Flink streaming application, the type of [state
backend](../state/state_backends.html) used
-will dictate the optimal memory configurations of your cluster.
+在部署 Flink 流处理应用时，可以根据 [State Backend](../state/state_backends.html)
的类型对集群的配置进行优化。
-### Heap state backend
+### Heap State Backend
-When running a stateless job or using a heap state backend
([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
-or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)), set
[managed memory](mem_setup.html#托管内存) to zero.
-This will ensure that the maximum amount of memory is allocated for user code
on the JVM.
+执行无状态作业或者使用 Heap State
Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend)
Review comment:
Broken link. Should be #memorystatebackend
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392805918 ## File path: docs/ops/memory/mem_migration.zh.md ## @@ -121,116 +117,105 @@ The following options are deprecated but if they are still used they will be int -Although, the network memory configuration has not changed too much it is recommended to verify its configuration. -It can change if other memory components have new sizes, e.g. the total memory which the network can be a fraction of. -See also [new detailed memory model](mem_detail.html). +尽管网络内存的配置参数没有发生太多变化，我们仍建议您检查其配置结果。 +网络内存的大小可能会受到其他内存部分大小变化的影响，例如总内存变化时，根据占比计算出的网络内存也可能发生变化。 +请参考[内存模型详解](mem_detail.html)。 -The container cut-off configuration options, [`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) -and [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min), have no effect for task manager processes anymore -but they still have the same semantics for the job manager process. See also [how to migrate container cut-off](#container-cut-off-memory). +容器切除（Cut-Off）内存相关的配置参数（[`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) +和 [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min)）将不再对 TaskExecutor 进程生效。 +对于 JobManager 进程，它们仍具有与此前相同的语义。 +请参考[如何升级容器切除内存](#容器切除cut-off内存)。 -## Total Memory (Previously Heap Memory) +## 总内存（原堆内存） -The previous options which were responsible for the total memory used by Flink are `taskmanager.heap.size` or `taskmanager.heap.mb`. -Despite their naming, they included not only JVM heap but also other off-heap memory components. The options have been deprecated. +在原本的内存配置方法中，用于指定用于 Flink 的总内存的配置参数是 `taskmanager.heap.size` 或 `taskmanager.heap.mb`。 +尽管这两个参数以“堆（Heap）”命名，实际上它们指定的内存既包含了 JVM 堆内存，也包含了其他堆外内存部分。 +这两个配置参数目前已被弃用。 -The Mesos integration also had a separate option with the same semantics: `mesos.resourcemanager.tasks.mem` which has also been removed. +Flink 在 Mesos 上还有另一个具有同样语义的配置参数 `mesos.resourcemanager.tasks.mem`，目前也已经被弃用。 -If the mentioned legacy options are used without specifying the corresponding new options, -they will be directly translated into the following new options: -* Total Flink memory ([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) for standalone deployments -* Total process memory ([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)) for containerized deployments (Yarn or Mesos) +如果配置了上述弃用的参数，同时又没有配置与之对应的新配置参数，那它们将按如下规则对应到新的配置参数。 +* 独立部署模式（Standalone Deployment）下：Flink 总内存（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)） +* 容器化部署模式（Containerized Deployement）下（Yarn、Mesos）：进程总内存（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)） -It is also recommended to use these new options instead of the legacy ones as they might be completely removed in the following releases. +建议您尽早使用新的配置参数取代启用的配置参数，它们在今后的版本中可能会被彻底移除。 -See also [how to configure total memory now](mem_setup.html#配置总内存). +请参考[如何配置总内存](mem_setup.html#配置总内存). -## JVM Heap Memory +## JVM 堆内存 -JVM heap memory previously consisted of the managed memory (if configured to be on-heap) and the rest -which included any other usages of heap memory. This rest was always implicitly derived as the remaining part of the total memory, -see also [how to migrate managed memory](#managed-memory). +此前，JVM 堆空间由托管内存（仅在配置为堆上时）及 Flink 用到的所有其他堆内存组成。 +这里的其他堆内存是由总内存减去所有其他非堆内存得到的。 +请参考[如何升级托管内存](#托管内存)。 -Now, if only *total Flink memory* or *total process memory* is configured, then the JVM heap is also derived as the rest of -what is left after subtracting all other components from the total memory, see also [how to configure total memory](mem_setup.html#配置总内存). +现在，如果仅配置了*Flink总内存*或*进程总内存*，JVM 的堆空间依然是根据总内存减去所有其他非堆内存得到的。 +请参考[如何配置总内存](mem_setup.html#配置总内存)。 -Additionally, you can now have more direct control over the JVM heap assigned to the operator tasks -([`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)), -see also [Task (Operator) Heap Memory](mem_setup.html#任务算子堆内存). -The JVM heap memory is also used by the heap state backends ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) -or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)) if it is chosen for streaming jobs. +此外，你现在可以更直接地控制用于任务和算子的 JVM 的堆内存（[`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)），详见[任务堆内存](mem_setup.html#任务算子堆内存)。 +如果流处理作业选择使用 Heap State Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) +或 [FsStateBackend](../state/state_backends.html#the-fsstatebackend)），那么它同样需要使用 JVM 堆内存。 Review comment: Broken link. --
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392810899 ## File path: docs/ops/memory/mem_migration.zh.md ## @@ -121,116 +117,105 @@ The following options are deprecated but if they are still used they will be int -Although, the network memory configuration has not changed too much it is recommended to verify its configuration. -It can change if other memory components have new sizes, e.g. the total memory which the network can be a fraction of. -See also [new detailed memory model](mem_detail.html). +尽管网络内存的配置参数没有发生太多变化，我们仍建议您检查其配置结果。 +网络内存的大小可能会受到其他内存部分大小变化的影响，例如总内存变化时，根据占比计算出的网络内存也可能发生变化。 +请参考[内存模型详解](mem_detail.html)。 -The container cut-off configuration options, [`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) -and [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min), have no effect for task manager processes anymore -but they still have the same semantics for the job manager process. See also [how to migrate container cut-off](#container-cut-off-memory). +容器切除（Cut-Off）内存相关的配置参数（[`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) +和 [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min)）将不再对 TaskExecutor 进程生效。 +对于 JobManager 进程，它们仍具有与此前相同的语义。 +请参考[如何升级容器切除内存](#容器切除cut-off内存)。 -## Total Memory (Previously Heap Memory) +## 总内存（原堆内存） -The previous options which were responsible for the total memory used by Flink are `taskmanager.heap.size` or `taskmanager.heap.mb`. -Despite their naming, they included not only JVM heap but also other off-heap memory components. The options have been deprecated. +在原本的内存配置方法中，用于指定用于 Flink 的总内存的配置参数是 `taskmanager.heap.size` 或 `taskmanager.heap.mb`。 +尽管这两个参数以“堆（Heap）”命名，实际上它们指定的内存既包含了 JVM 堆内存，也包含了其他堆外内存部分。 +这两个配置参数目前已被弃用。 -The Mesos integration also had a separate option with the same semantics: `mesos.resourcemanager.tasks.mem` which has also been removed. +Flink 在 Mesos 上还有另一个具有同样语义的配置参数 `mesos.resourcemanager.tasks.mem`，目前也已经被弃用。 -If the mentioned legacy options are used without specifying the corresponding new options, -they will be directly translated into the following new options: -* Total Flink memory ([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) for standalone deployments -* Total process memory ([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)) for containerized deployments (Yarn or Mesos) +如果配置了上述弃用的参数，同时又没有配置与之对应的新配置参数，那它们将按如下规则对应到新的配置参数。 +* 独立部署模式（Standalone Deployment）下：Flink 总内存（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)） +* 容器化部署模式（Containerized Deployement）下（Yarn、Mesos）：进程总内存（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)） -It is also recommended to use these new options instead of the legacy ones as they might be completely removed in the following releases. +建议您尽早使用新的配置参数取代启用的配置参数，它们在今后的版本中可能会被彻底移除。 -See also [how to configure total memory now](mem_setup.html#配置总内存). +请参考[如何配置总内存](mem_setup.html#配置总内存). -## JVM Heap Memory +## JVM 堆内存 -JVM heap memory previously consisted of the managed memory (if configured to be on-heap) and the rest -which included any other usages of heap memory. This rest was always implicitly derived as the remaining part of the total memory, -see also [how to migrate managed memory](#managed-memory). +此前，JVM 堆空间由托管内存（仅在配置为堆上时）及 Flink 用到的所有其他堆内存组成。 +这里的其他堆内存是由总内存减去所有其他非堆内存得到的。 +请参考[如何升级托管内存](#托管内存)。 -Now, if only *total Flink memory* or *total process memory* is configured, then the JVM heap is also derived as the rest of -what is left after subtracting all other components from the total memory, see also [how to configure total memory](mem_setup.html#配置总内存). +现在，如果仅配置了*Flink总内存*或*进程总内存*，JVM 的堆空间依然是根据总内存减去所有其他非堆内存得到的。 +请参考[如何配置总内存](mem_setup.html#配置总内存)。 -Additionally, you can now have more direct control over the JVM heap assigned to the operator tasks -([`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)), -see also [Task (Operator) Heap Memory](mem_setup.html#任务算子堆内存). -The JVM heap memory is also used by the heap state backends ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) -or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)) if it is chosen for streaming jobs. +此外，你现在可以更直接地控制用于任务和算子的 JVM 的堆内存（[`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)），详见[任务堆内存](mem_setup.html#任务算子堆内存)。 +如果流处理作业选择使用 Heap State Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) +或 [FsStateBackend](../state/state_backends.html#the-fsstatebackend)），那么它同样需要使用 JVM 堆内存。 -A part of the JVM heap is now always reserved for Flink framework -
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh]
Translate memory configuration docs into Chinese.
URL: https://github.com/apache/flink/pull/11401#discussion_r392804787
##
File path: docs/ops/memory/mem_migration.zh.md
##
@@ -22,67 +22,63 @@ specific language governing permissions and limitations
under the License.
-->
-The [memory setup of task managers](mem_setup.html) has changed a lot with the
1.10 release. Many configuration options
-were removed or their semantics changed. This guide will help you to migrate
the memory configuration from Flink
-[<=
*1.9*](https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/mem_setup.html)
to >= *1.10*.
+在 1.10 版本中，Flink 的 [TaskExecutor 内存配置方法](mem_setup.html)发生了较大的变化。
+部分配置参数被移除了，或是语义上发生了变化。
+本篇升级指南将介绍如何将 [*Flink 1.9
及以前版本*](https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/mem_setup.html)的内存配置升级到
*Flink 1.10 及以后版本*。
* toc
{:toc}
- Warning: It is important to review this guide because the
legacy and new memory configuration can
- result in different sizes of memory components. If you try to reuse your
Flink configuration from older versions
- before 1.10, it can result in changes to the behavior, performance or even
configuration failures of your application.
+ 注意： 请仔细阅读本篇升级指南。
+ 使用原本的和新的内存配制方法可能会使内存组成部分具有截然不同的大小。
+ 未经调整直接沿用 Flink 1.10 以前版本的配置文件，可能导致应用的行为、性能发生变化，甚至造成应用执行失败。
-Note Before version *1.10*, Flink did
not require that memory related options are set at all
-as they all had default values. The [new memory
configuration](mem_setup.html#配置总内存) requires
-that at least one subset of the following options is configured explicitly,
otherwise the configuration will fail:
+提示 在 *1.10* 版本之前，Flink
不要求用户一定要配置内存相关的参数，因为这些参数都具有默认值。
+[新的内存配置](mem_setup.html#配置总内存)要求用户至少指定下列配置参数（或参数组合）的其中之一，否则 Flink 将无法启动。
*
[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)
*
[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)
-*
[`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)
and
[`taskmanager.memory.managed.size`](../config.html#taskmanager-memory-managed-size)
+*
[`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)
和
[`taskmanager.memory.managed.size`](../config.html#taskmanager-memory-managed-size)
-The [default `flink-conf.yaml`](#default-configuration-in-flink-confyaml)
shipped with Flink sets
[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)
-to make the default memory configuration consistent.
+Flink 自带的[默认 flink-conf.yaml](#flink-confyaml-中的默认配置) 文件指定了
[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)，以便与此前的行为保持一致。
-This
[spreadsheet](https://docs.google.com/spreadsheets/d/1mJaMkMPfDJJ-w6nMXALYmTc4XxiV30P5U7DzgwLkSoE)
can also help
-to evaluate and compare the results of the legacy and new memory computations.
+可以使用这张[电子表格](https://docs.google.com/spreadsheets/d/1mJaMkMPfDJJ-w6nMXALYmTc4XxiV30P5U7DzgwLkSoE)来估算和比较原本的和新的内存配置下的计算结果。
-## Changes in Configuration Options
+## 配置参数变化
-This chapter shortly lists all changes to Flink's memory configuration options
introduced with the *1.10* release.
-It also references other chapters for more details about migrating to the new
configuration options.
+本节简要列出了 *Flink 1.10* 引入的配置参数变化，并援引其他章节中关于如何升级到新配置参数的相关描述。
-The following options are completely removed. If they are still used, they
will be ignored.
+下列配置参数已被彻底移除，配置它们将不会产生任何效果。
-Removed option
-Note
+移除的配置参数
+备注
taskmanager.memory.fraction
-Check the description of the new option taskmanager.memory.managed.fraction.
-The new option has different semantics and the value of the
deprecated option usually has to be adjusted.
-See also how to migrate managed
memory.
+请参考新配置参数 taskmanager.memory.managed.fraction
的相关描述。
Review comment:
Broken link. Redundant ")".
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392803199 ## File path: docs/ops/memory/mem_trouble.zh.md ## @@ -27,48 +27,43 @@ under the License. ## IllegalConfigurationException -If you see an *IllegalConfigurationException* thrown from *TaskExecutorProcessUtils*, it usually indicates -that there is either an invalid configuration value (e.g. negative memory size, fraction that is greater than 1, etc.) -or configuration conflicts. Check the documentation chapters related to the [memory components](mem_detail.html) -mentioned in the exception message. +如果遇到从 *TaskExecutorProcessUtils* 抛出的 *IllegalConfigurationException* 异常，这通常说明您的配置参数中存在无效值（例如内存大小为负数、占比大于 1 等）或者配置冲突。 +请根据异常信息，确认[内存模型详解](mem_detail.html)中与出错的内存部分对应章节的内容。 ## OutOfMemoryError: Java heap space -The exception usually indicates that the JVM heap is too small. You can try to increase the JVM heap size -by increasing [total memory](mem_setup.html#配置总内存) or [task heap memory](mem_setup.html#任务算子堆内存). +该异常说明 JVM 的堆空间过小。 +可以通过增大[总内存](mem_setup.html#配置总内存)或[任务堆内存](mem_setup.html#任务算子堆内存)的方法来增大 JVM 堆空间。 -Note You can also increase the [framework heap memory](mem_detail.html#框架内存) but this option -is advanced and should only be changed if you are sure that the Flink framework itself needs more memory. +提示 也可以增大[框架堆内存](mem_detail.html#框架内存)。这是一个进阶配置，只有在确认是 Flink 框架自身需要更多内存时才应该去调整。 ## OutOfMemoryError: Direct buffer memory -The exception usually indicates that the JVM *direct memory* limit is too small or that there is a *direct memory leak*. -Check whether user code or other external dependencies use the JVM *direct memory* and that it is properly accounted for. -You can try to increase its limit by adjusting [direct off-heap memory](mem_detail.html). -See also [how to configure off-heap memory](mem_setup.html#配置堆外内存直接内存或本地内存)) and -the [JVM arguments](mem_detail.html#jvm-参数) which Flink sets. +该异常通常说明 JVM 的*直接内存*限制过小，或者存在*直接内存泄漏（Direct Memory Leak）*。 +请确认用户代码及外部依赖中是否使用了 JVM *直接内存*，以及如果使用了直接内存，是否配置了足够的内存空间。 +可以通过调整[堆外内存](mem_detail.html)来增大直接内存限制。 +请同时参考[如何配置堆外内存](mem_setup.html#配置堆外内存直接内存或本地内存))以及 Flink 设置的 [JVM 参数](mem_detail.html#jvm-参数)。 ## OutOfMemoryError: Metaspace -The exception usually indicates that [JVM metaspace limit](mem_detail.html#jvm-参数) is configured too small. -You can try to increase the [JVM metaspace option](../config.html#taskmanager-memory-jvm-metaspace-size). +该异常说明 [JVM Metaspace 限制](mem_detail.html#jvm-参数)过小。 +可以尝试调整 [JVM Metaspace 参数](../config.html#taskmanager-memory-jvm-metaspace-size)。 ## IOException: Insufficient number of network buffers -The exception usually indicates that the size of the configured [network memory](mem_detail.html) -is not big enough. You can try to increase the *network memory* by adjusting the following options: +该异常通常说明[网络内存](mem_detail.html)过小。 Review comment: #受限的等比内存部分 could be better. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese.
KarmaGYZ commented on a change in pull request #11401: [FLINK-15991][doc][zh] Translate memory configuration docs into Chinese. URL: https://github.com/apache/flink/pull/11401#discussion_r392809260 ## File path: docs/ops/memory/mem_migration.zh.md ## @@ -121,116 +117,105 @@ The following options are deprecated but if they are still used they will be int -Although, the network memory configuration has not changed too much it is recommended to verify its configuration. -It can change if other memory components have new sizes, e.g. the total memory which the network can be a fraction of. -See also [new detailed memory model](mem_detail.html). +尽管网络内存的配置参数没有发生太多变化，我们仍建议您检查其配置结果。 +网络内存的大小可能会受到其他内存部分大小变化的影响，例如总内存变化时，根据占比计算出的网络内存也可能发生变化。 +请参考[内存模型详解](mem_detail.html)。 -The container cut-off configuration options, [`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) -and [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min), have no effect for task manager processes anymore -but they still have the same semantics for the job manager process. See also [how to migrate container cut-off](#container-cut-off-memory). +容器切除（Cut-Off）内存相关的配置参数（[`containerized.heap-cutoff-ratio`](config.html#containerized-heap-cutoff-ratio) +和 [`containerized.heap-cutoff-min`](config.html#containerized-heap-cutoff-min)）将不再对 TaskExecutor 进程生效。 +对于 JobManager 进程，它们仍具有与此前相同的语义。 +请参考[如何升级容器切除内存](#容器切除cut-off内存)。 -## Total Memory (Previously Heap Memory) +## 总内存（原堆内存） -The previous options which were responsible for the total memory used by Flink are `taskmanager.heap.size` or `taskmanager.heap.mb`. -Despite their naming, they included not only JVM heap but also other off-heap memory components. The options have been deprecated. +在原本的内存配置方法中，用于指定用于 Flink 的总内存的配置参数是 `taskmanager.heap.size` 或 `taskmanager.heap.mb`。 +尽管这两个参数以“堆（Heap）”命名，实际上它们指定的内存既包含了 JVM 堆内存，也包含了其他堆外内存部分。 +这两个配置参数目前已被弃用。 -The Mesos integration also had a separate option with the same semantics: `mesos.resourcemanager.tasks.mem` which has also been removed. +Flink 在 Mesos 上还有另一个具有同样语义的配置参数 `mesos.resourcemanager.tasks.mem`，目前也已经被弃用。 -If the mentioned legacy options are used without specifying the corresponding new options, -they will be directly translated into the following new options: -* Total Flink memory ([`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)) for standalone deployments -* Total process memory ([`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)) for containerized deployments (Yarn or Mesos) +如果配置了上述弃用的参数，同时又没有配置与之对应的新配置参数，那它们将按如下规则对应到新的配置参数。 +* 独立部署模式（Standalone Deployment）下：Flink 总内存（[`taskmanager.memory.flink.size`](../config.html#taskmanager-memory-flink-size)） +* 容器化部署模式（Containerized Deployement）下（Yarn、Mesos）：进程总内存（[`taskmanager.memory.process.size`](../config.html#taskmanager-memory-process-size)） -It is also recommended to use these new options instead of the legacy ones as they might be completely removed in the following releases. +建议您尽早使用新的配置参数取代启用的配置参数，它们在今后的版本中可能会被彻底移除。 -See also [how to configure total memory now](mem_setup.html#配置总内存). +请参考[如何配置总内存](mem_setup.html#配置总内存). -## JVM Heap Memory +## JVM 堆内存 -JVM heap memory previously consisted of the managed memory (if configured to be on-heap) and the rest -which included any other usages of heap memory. This rest was always implicitly derived as the remaining part of the total memory, -see also [how to migrate managed memory](#managed-memory). +此前，JVM 堆空间由托管内存（仅在配置为堆上时）及 Flink 用到的所有其他堆内存组成。 +这里的其他堆内存是由总内存减去所有其他非堆内存得到的。 +请参考[如何升级托管内存](#托管内存)。 -Now, if only *total Flink memory* or *total process memory* is configured, then the JVM heap is also derived as the rest of -what is left after subtracting all other components from the total memory, see also [how to configure total memory](mem_setup.html#配置总内存). +现在，如果仅配置了*Flink总内存*或*进程总内存*，JVM 的堆空间依然是根据总内存减去所有其他非堆内存得到的。 +请参考[如何配置总内存](mem_setup.html#配置总内存)。 -Additionally, you can now have more direct control over the JVM heap assigned to the operator tasks -([`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)), -see also [Task (Operator) Heap Memory](mem_setup.html#任务算子堆内存). -The JVM heap memory is also used by the heap state backends ([MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) -or [FsStateBackend](../state/state_backends.html#the-fsstatebackend)) if it is chosen for streaming jobs. +此外，你现在可以更直接地控制用于任务和算子的 JVM 的堆内存（[`taskmanager.memory.task.heap.size`](../config.html#taskmanager-memory-task-heap-size)），详见[任务堆内存](mem_setup.html#任务算子堆内存)。 +如果流处理作业选择使用 Heap State Backend（[MemoryStateBackend](../state/state_backends.html#the-memorystatebackend) +或 [FsStateBackend](../state/state_backends.html#the-fsstatebackend)），那么它同样需要使用 JVM 堆内存。 -A part of the JVM heap is now always reserved for Flink framework -
