I am not aware of any autoscaler/operator like functionality that works on
EMR/YARN at the moment.
It is probably easier to migrate to Kubernetes, or improve the Flink
Kubernetes Operator itself to be able to manage YARN clusters than to
improve all these things for yarn directly :)
Gyula
On Sun
Hi Gyula/Team,
Thanks for your response. I see that the above one works on Kubernetes, but all
our applications are currently running on EMR Yarn and looking to integrate
with EMR-managed scaling with adaptive scheduler.
Do we have any features or operators which we can use to run on Yarn simila
Hi Madan,
With reactive mode you need to build a completely custom auto scaling
logic, it can work but it takes considerable effort.
Instead I recommend using the Flink Kubernetes Operator which now contains
the first version of an actual autoscaler module that collects metrics and
scales Flink j
Hello Team,I would like to understand auto scaling on EMR using either reactive
mode or adaptive scheduler with custom or managed scaling.Can some one help me
on this.
Regards,Madan
