Re: Scheduling task slots in round-robin

I have opened a PR with my initial implementation, it seems to work well on the cluster at least for fairly simple jobs (haven't tried with iterations). It would be great to get some feedback before I start diving into the scheduler tests :) But no need

Re: Scheduling task slots in round-robin

Hi Till, Thanks for the pointers, I started looking into this and it does not seem to be too complicated add a new strategy :) I will try to put together a PR but I would greatly appreciate if you could check it out once I'm done as I don't have too much experience with these components. Cheers,

Re: Scheduling task slots in round-robin

Hi Gyula, the scheduler actually deploys independent tasks in a round-robin fashion across the cluster. So for example, your source sub tasks should be spread evenly. However, whenever a sub-task has an input, it tries to deploy this task on the same machine as one of the input sub-tasks (preferre

Re: Scheduling task slots in round-robin

Thanks! I found this PR already but seemed to be completely outdated :) Maybe it's worth restarting this discussion. Gyula Chesnay Schepler ezt írta (időpont: 2016. jún. 13., H, 14:58): > FLINK-1003 may be related. > > On 13.06.2016 12:46, Gyula Fóra wrote: > > Hey, > > > > The Flink schedulin

Re: Scheduling task slots in round-robin

FLINK-1003 may be related. On 13.06.2016 12:46, Gyula Fóra wrote: Hey, The Flink scheduling mechanism has become quite a bit of a pain lately for us when trying to schedule IO heavy streaming jobs. And by IO heavy I mean it has a fairly large state that is being continuously updated/read. The

Scheduling task slots in round-robin

Hey, The Flink scheduling mechanism has become quite a bit of a pain lately for us when trying to schedule IO heavy streaming jobs. And by IO heavy I mean it has a fairly large state that is being continuously updated/read. The main problem is that the scheduled task slots are not evenly distribu