Ok thanks ! All that’s left is to wait then.
B.R. From: ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] On Behalf Of Stephan Ewen Sent: jeudi 4 février 2016 11:19 To: user@flink.apache.org Subject: Re: Internal buffers supervision and yarn vCPUs Concerning the first question: What you are looking for is backpressure monitoring. If a task cannot push its data to the next task, it is backpressured. This pull request adds a first version of backpressure monitoring: https://github.com/apache/flink/pull/1578 We will try and get it merged soon! On Thu, Feb 4, 2016 at 11:03 AM, Robert Metzger <rmetz...@apache.org<mailto:rmetz...@apache.org>> wrote: Hi Gwen, let me answer the second question: There is a JIRA to reintroduce the configuration parameter: https://issues.apache.org/jira/browse/FLINK-2213. I will try to get a fix for this into the 1.0 release. I think I removed back then because users were unable to define the number of vcores independently of the number of slots ... and too many users were running into issues with the yarn scheduler (containers were not started because there were no CPU resources available anymore). On Thu, Feb 4, 2016 at 10:56 AM, Gwenhael Pasquiers <gwenhael.pasqui...@ericsson.com<mailto:gwenhael.pasqui...@ericsson.com>> wrote: Hi, I’ve got two more questions on different topic… First one : Is there a way to monitor the buffers status. In order to find bottleneck in our application we though it could be usefull to be able to have a look at the different exchange buffers’ status. To know if they are full (or as an example if a mapper had to wait before being able to push it’s data into the buffer). That way we can know where the bottleneck is. Second one : On type of resources on yarn is vCPU. In flink 0.8 there was a “-tmc” argument that allowed to specify the number of vCPU per task manager. We cannot find it anymore. Was it removed ? Is there another way to set the number of vCPU. Or did it became useless ? Thanks in advance. Gwen’
