zhuzhurk commented on a change in pull request #10427: [FLINK-14566] Enable to get/set whether an operator uses managed memory URL: https://github.com/apache/flink/pull/10427#discussion_r354196126
########## File path: flink-core/src/main/java/org/apache/flink/api/dag/Transformation.java ########## @@ -136,6 +138,14 @@ public static int getNewNodeId() { */ private ResourceSpec preferredResources = ResourceSpec.DEFAULT; + /** + * This weight indicates how much this transformation relies on managed memory, so that + * transformation highly relies on managed memory would be able to acquire more managed + * memory in runtime (linear association). Note that it only works in cases of UNKNOWN + * resources. + */ + private int managedMemoryWeight = DEFAULT_MANAGED_MEMORY_WEIGHT; Review comment: > I am +1 to default `don't need managed memory`. > Note it is manage memory, only way is requiring memory from memory manager explicitly. It is different from other resources. > > And image user write an operator which using manage memory, and he believe there is only one operator to use manage memory, that is his operator. But if he use DataStream/DataSet api, whatever operators including map/source... these operators will rob his memories. RocksDB StateBackends also use managed memory. Making it 0 by default would break existing streaming jobs if users do not explicitly set the managed memory weight (and they even cannot set the weights because it is not a public interface to external users). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services