Re: How to distribute subtasks evenly across taskmanagers?

2018-12-05 Thread Till Rohrmann
Hi Sunny, this is a current limitation of Flink's scheduling. We are currently working on extending Flinks scheduling mechanism [1] which should also help with solving this problem. At the moment, I recommend using the per-job mode so that you have a single cluster per job. [1] https://issues.apa

How to distribute subtasks evenly across taskmanagers?

2018-12-04 Thread Sunny Yun
Why does Flink do resource management by only slots, not by TaskManagers and slots? If there are one Flink cluster to submit multiple jobs, how do I make JobManager to distribute subtasks evenly to all TaskManagers? Now, JobManager treats the slots globally, some jobs' operators are assigned only