xintongsong commented on issue #9760: [FLINK-13982][runtime] Implement memory calculation logics URL: https://github.com/apache/flink/pull/9760#issuecomment-538212636 @tillrohrmann, I agree that the memory configurations can be modeled as a linear programming problem, but I'm not sure whether we should do this. I'm in favor of derive the memory sizes manually, like how it is done currently with a if-else branches. I have some concerns on using known linear programming solvers. - We are limited by the double type precision, and cannot use BigDecimal in case of rounding errors. - If there is a `NoFeasibleSolutionException`, we may not be able to provide helpful information to help the user fix their configurations. E.g., which values are not adding up, and which values can potentially be increased/decreased to fix the problem. On the other hand, manually calculating the memory sizes (one after another) may not solve the exactly same linear programming problem (it may not explore the entire solution space due to early deciding some of the memory sizes). But maybe this is not a bad thing. I think it would be easier for user to understand how a particular memory size is calculated (less dependencies on others), and how to change/ fix it in case of conflicts. I think it would be good that given the explicitly configured values, we try to find a feasible solution by tuning the none specified values. But I think it's not necessary that as long as there's a feasible solution we can always find it, especially at the cost of the two concerns mentioned above. What do you think?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
