Hi all,
I and Fan Rui(CC’ed) created the FLIP-370[1] to support balanced tasks scheduling. The current strategy of Flink to deploy tasks sometimes leads some TMs(TaskManagers) to have more tasks while others have fewer tasks, resulting in excessive resource utilization at some TMs that contain more tasks and becoming a bottleneck for the entire job processing. Developing strategies to achieve task load balancing for TMs and reducing job bottlenecks becomes very meaningful. The raw design and discussions could be found in the Flink JIRA[2] and Google doc[3]. We really appreciate Zhu Zhu(CC’ed) for providing some valuable help and suggestions in advance. Please refer to the FLIP[1] document for more details about the proposed design and implementation. We welcome any feedback and opinions on this proposal. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-370%3A+Support+Balanced+Tasks+Scheduling [2] https://issues.apache.org/jira/browse/FLINK-31757 [3] https://docs.google.com/document/d/14WhrSNGBdcsRl3IK7CZO-RaZ5KXU2X1dWqxPEFr3iS8 Best, Yuepeng Pan
