[ https://issues.apache.org/jira/browse/FLINK-34471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17819578#comment-17819578 ]
Maximilian Michels edited comment on FLINK-34471 at 2/22/24 10:18 AM: ---------------------------------------------------------------------- Thanks! That would be very helpful – I don’t mind at all. I realized point to point connections might be a bit tricky when the parallelism between tasks changes from being equal to being different because at runtime we then switch to a different partitioner. So worst case we could run out of network buffers in this scenario. I’m curious how you want to solve this. We probably need to add extra buffers to account for this edge case. was (Author: mxm): Thanks! That would be very helpful – I don’t mind at all. I realized point to point connections might be a bit tricky when the parallelism between tasks changes from being equal to changing because at runtime we then switch to a different partitioner. So worst case we could run out of network buffers in this scenario. I’m curious how you want to solve this. We probably need to add extra buffers to account for this edge case. > Tune network memory as part of Autoscaler Memory Tuning > ------------------------------------------------------- > > Key: FLINK-34471 > URL: https://issues.apache.org/jira/browse/FLINK-34471 > Project: Flink > Issue Type: Improvement > Components: Autoscaler > Reporter: Rui Fan > Assignee: Maximilian Michels > Priority: Major > > Design doc: > https://docs.google.com/document/d/19HYamwMaYYYOeH3NRbk6l9P-bBLBfgzMYjfGEPWEbeo/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.20.10#820010)