Nishchay Malhotra, what scheduler are you using? Also, what are the settings
for each queue?
From: Billy Watson
To: nishchay malhotra
Cc: "common-u...@hadoop.apache.org"
Sent: Tuesday, January 30, 2018 9:47 AM
Subject: Re: Regarding containers not launching
Is your j
Also, is there anything interesting in the yarn scheduler logs? Something
about scheduling being skipped?
On Wed, Jan 31, 2018 at 05:16 Billy Watson wrote:
> Ok, and your container settings?
>
> On Wed, Jan 31, 2018 at 02:38 nishchay malhotra <
> nishchay.malht...@gmail.com> wrote:
>
>> yes m
Ok, and your container settings?
On Wed, Jan 31, 2018 at 02:38 nishchay malhotra
wrote:
> yes my job has about 160,000 maps and my cluster not getting fully
> utilized around 6000 maps ran for 2 hrs and then I killed the job. At any
> point of time only 40 containers are running thats just 11% o
yes my job has about 160,000 maps and my cluster not getting fully utilized
around 6000 maps ran for 2 hrs and then I killed the job. At any point of
time only 40 containers are running thats just 11% of my cluster capacity.
{
"classification": "mapred-site",
"properties": {
"mapredu
Is your job able to use more containers, I.e. does your job have tasks
waiting or are all tasks in progress?
William Watson
On Tue, Jan 30, 2018 at 1:56 AM, nishchay malhotra <
nishchay.malht...@gmail.com> wrote:
> What should I be looking for if my 24-node cluster in not launching enough
> con
What should I be looking for if my 24-node cluster in not launching enough
containers?
only 40/288 cores are used and 87GB/700GB is memory is used.
Yarn.nodemanager memory/core conf look good. And so do container
memory/core conf.
Thanks
Nishchay Malhotra