Re: YARN Reserved Memory

2016-12-07 Thread Michael Pisula
Hi Stefan, thanks for the fast feedback. Updating to a newer YARN Version is most certainly something that would benefit us in many different areas (the issues with the HA mode being the most important of them), however at the moment we are not able to update to a newer version. If that is another

Re: YARN Reserved Memory

2016-12-07 Thread Stefan Richter
Hi, did you observe the problem only under YARN 2.4.0? IIRC this version of YARN has some problems that can also lead to issues with Flink’s HA mode, and I would encourage you to upgrade YARN to 2.5 or higher. On a different note, there have been several improvements that we will release in Fli

YARN Reserved Memory

2016-12-07 Thread Michael Pisula
Hi Guys, We are having a slight issue using Flink 1.1.3 (we also observed the problem with 1.0.2) in Yarn 2.4.0. Whenever a TaskManager restarts, YARN seems to reserve memory during the TaskManager restart, and not free the memory again. We are using a CapacityScheduler with 2 queues, where the qu