Hi community,
In the past releases, we have been working on refactoring Flink's scheduler
with the goal of making the scheduler extensible [1]. We have rolled out
most of the intended refactoring in Flink 1.10, and we think it is now time
to leverage our newly introduced abstractions to implement a new resource
optimized scheduling strategy: Pipelined Region Scheduling.
This scheduling strategy aims at:
* avoidance of resource deadlocks when running batch jobs
* tunable with respect to resource consumption and throughput
More details can be found in the Wiki [2]. We are looking forward to your
feedback.
Best,
Zhu Zhu & Gary
[1] https://issues.apache.org/jira/browse/FLINK-10429
[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-119+Pipelined+Region+Scheduling
