Yarn Over-allocating Containers

2014-09-12 Thread praveen seluka
Hi all Am seeing a strange issue in Spark on Yarn(Stable). Let me know if known, or am missing something as it looks very fundamental. Launch a Spark job with 2 Containers. addContainerRequest called twice and then calls allocate to AMRMClient. This will get 2 Containers allocated. Fine as of

Re: Yarn Over-allocating Containers

2014-09-12 Thread Sandy Ryza
Hi Praveen, I believe you are correct. I noticed this a little while ago and had a fix for it as part of SPARK-1714, but that's been delayed. I'll look into this a little deeper and file a JIRA. -Sandy On Thu, Sep 11, 2014 at 11:44 PM, praveen seluka praveen.sel...@gmail.com wrote: Hi all