Re: Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-10 Thread Ravi Prakash
um-capacity >> >> Maximum queue capacity in percentage (%) as a float. This limits the >> *elasticity* for applications in the queue. Defaults to -1 which >> disables it. >> >> >> >> 2. Preemption of containers. >> >> >> >>

Re: Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-10 Thread Rafał Radecki
From:* Rafał Radecki [mailto:radecki.ra...@gmail.com] > *Sent:* 10 November 2016 17:26 > *To:* Bibinchundatt > *Cc:* Ravi Prakash; user > > *Subject:* Re: Yarn 2.7.3 - capacity scheduler container allocation to > nodes? > > > > We have 4 nodes and 4 large (~30GB each tasks), add

RE: Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-10 Thread Bibinchundatt
of containers. Regards Bibin From: Rafał Radecki [mailto:radecki.ra...@gmail.com] Sent: 10 November 2016 17:26 To: Bibinchundatt Cc: Ravi Prakash; user Subject: Re: Yarn 2.7.3 - capacity scheduler container allocation to nodes? We have 4 nodes and 4 large (~30GB each tasks), additionally we have

Re: Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-10 Thread Rafał Radecki
usecase specifically for per node allocation based on > percentage? > > > > > > *From:* Rafał Radecki [mailto:radecki.ra...@gmail.com] > *Sent:* 10 November 2016 14:59 > *To:* Ravi Prakash > *Cc:* user > *Subject:* Re: Yarn 2.7.3 - capacity scheduler container allocation

RE: Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-10 Thread Bibinchundatt
percentage? From: Rafał Radecki [mailto:radecki.ra...@gmail.com] Sent: 10 November 2016 14:59 To: Ravi Prakash Cc: user Subject: Re: Yarn 2.7.3 - capacity scheduler container allocation to nodes? Hi Ravi. I did not specify labels this time ;) I just created two queues as it is visible in the

Re: Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-09 Thread Rafał Radecki
Hi Ravi. I did not specify labels this time ;) I just created two queues as it is visible in the configuration. Overall queues work but allocation of jobs is different then expected by me as I wrote at the beginning. BR, Rafal. 2016-11-10 2:48 GMT+01:00 Ravi Prakash : > Hi Rafal! > > Have you b

Re: Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-09 Thread Ravi Prakash
Hi Rafal! Have you been able to launch the job successfully first without configuring node-labels? Do you really need node-labels? How much total memory do you have on the cluster? Node labels are usually for specifying special capabilities of the nodes (e.g. some nodes could have GPUs and your ap

Yarn 2.7.3 - capacity scheduler container allocation to nodes?

2016-11-09 Thread Rafał Radecki
Hi All. I have a 4 node cluster on which I run yarn. I created 2 queues "long" and "short", first with 70% resource allocation, the second with 30% allocation. Both queues are configured on all available nodes by default. My memory for yarn per node is ~50GB. Initially I thought that when I will