Hi all,
First time poster, so go easy on me :)
What is Flink's story for accommodating task workloads with vastly
disparate resource requirements: e.g. some require very little CPU and RAM,
while others require quite a lot.
Our current strategy is to bundle resource-intensive tasks and send them
Hi Chad,
If I understand correctly, the scenarios you talked about are running batch
jobs, right?
At the moment (Flink 1.8 and earlier), Flink does not differentiate
different working load of tasks. It uses a slot-sharing approach[1] to
balance workloads among workers. The general idea is to put
Hi Chad,
Just as Xintong said, fine grained resource management has not been
introduced to flink. And i think it is the elegant solution for
your scenario. Task managers with different resource specification will be
allocated and started by Yarn/k8s resource manager according to your
operator reso
Thanks for the great feedback.
Just as Xintong said, fine grained resource management has not been
> introduced to flink. And i think it is the elegant solution for
> your scenario. Task managers with different resource specification will be
> allocated and started by Yarn/k8s resource manager acc
