[
https://issues.apache.org/jira/browse/FLINK-34152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822667#comment-17822667
]
Yang LI commented on FLINK-34152:
---------------------------------
Hi [~mxm] , Here's some feedback that might be slightly outside the scope: I've
conducted tests with a few custom enhancements aimed at:
# Protecting memory during scale-down operations to prevent the Flink cluster
from experiencing Out of Memory (OOM) issues.
# Implementing a minor improvement to refine the scaling decisions following
each operator's rescaling, ensuring there are no unused slots post-rescaling.
After reviewing the memory tuning feature, I believe my memory protection
enhancement may no longer be necessary. However, I've still pushed the minor
improvement to eliminate any unused slots.
Can you take a look on it https://issues.apache.org/jira/browse/FLINK-34563 and
[https://github.com/apache/flink-kubernetes-operator/pull/787] ? And tell me if
you think it's making sense, thanks :)
> Tune TaskManager memory
> -----------------------
>
> Key: FLINK-34152
> URL: https://issues.apache.org/jira/browse/FLINK-34152
> Project: Flink
> Issue Type: Sub-task
> Components: Autoscaler, Kubernetes Operator
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Labels: pull-request-available
> Fix For: kubernetes-operator-1.8.0
>
>
> The current autoscaling algorithm adjusts the parallelism of the job task
> vertices according to the processing needs. By adjusting the parallelism, we
> systematically scale the amount of CPU for a task. At the same time, we also
> indirectly change the amount of memory tasks have at their dispense. However,
> there are some problems with this.
> # Memory is overprovisioned: On scale up we may add more memory than we
> actually need. Even on scale down, the memory / cpu ratio can still be off
> and too much memory is used.
> # Memory is underprovisioned: For stateful jobs, we risk running into
> OutOfMemoryErrors on scale down. Even before running out of memory, too
> little memory can have a negative impact on the effectiveness of the scaling.
> We lack the capability to tune memory proportionally to the processing needs.
> In the same way that we measure CPU usage and size the tasks accordingly, we
> need to evaluate memory usage and adjust the heap memory size.
> https://docs.google.com/document/d/19GXHGL_FvN6WBgFvLeXpDABog2H_qqkw1_wrpamkFSc/edit
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
