in cluster configuration JSON).
>>
>>
>>
>> Thanks again to Sunil and Shubh (and my colleague, York) for the helpful
>> guidance!
>>
>>
>>
>> Take care,
>>
>> -Jeff
>>
>>
>>
>> *From:* Shubh hadoopExp [m
idance!
>
>
>
> Take care,
>
> -Jeff
>
>
>
> *From:* Shubh hadoopExp [mailto:shubhhadoop...@gmail.com]
> *Sent:* Wednesday, May 25, 2016 11:08 PM
> *To:* Guttadauro, Jeff <jeff.guttada...@here.com>
> *Cc:* Sunil Govind <sunil.gov...@gmail.com>; user@hadoo
e, York) for the helpful
>> guidance!
>>
>> Take care,
>> -Jeff
>>
>> From: Shubh hadoopExp [mailto:shubhhadoop...@gmail.com
>> <mailto:shubhhadoop...@gmail.com>]
>> Sent: Wednesday, May 25, 2016 11:08 PM
>> To: Guttadauro,
Yes, that’s correct, Shubh. Thanks again…
From: Shubh hadoopExp [mailto:shubhhadoop...@gmail.com]
Sent: Saturday, May 28, 2016 3:20 AM
To: Guttadauro, Jeff <jeff.guttada...@here.com>
Cc: user@hadoop.apache.org
Subject: Re: YARN cluster underutilization
Hey
Thats pretty good. So by ch
l.com]
> Sent: Wednesday, May 25, 2016 11:08 PM
> To: Guttadauro, Jeff <jeff.guttada...@here.com>
> Cc: Sunil Govind <sunil.gov...@gmail.com>; user@hadoop.apache.org
> Subject: Re: YARN cluster underutilization
>
> Hey,
>
> OFFSWITCH allocation means if the data localit
;jeff.guttada...@here.com>
Cc: Sunil Govind <sunil.gov...@gmail.com>; user@hadoop.apache.org
Subject: Re: YARN cluster underutilization
Hey,
OFFSWITCH allocation means if the data locality is maintained or not. It has no
relation with heartbeat! Heartbeat is just used to clear the pipelining of
ou suggest any other knobs to turn to help RM
> handle it?
>
> Thanks again for all your help, Sunil!
>
> From: Sunil Govind [mailto:sunil.gov...@gmail.com]
> Sent: Wednesday, May 25, 2016 1:07 PM
> To: Guttadauro, Jeff <jeff.guttada...@here.com>; user@hadoop.
dnesday, May 25, 2016 1:07 PM
>
>
> *To:* Guttadauro, Jeff <jeff.guttada...@here.com>; user@hadoop.apache.org
> *Subject:* Re: YARN cluster underutilization
>
>
>
> Hi Jeff,
>
>
>
> I do see the yarn.resourcemanager.nodemanagers.heartbeat-int
RM
handle it?
Thanks again for all your help, Sunil!
From: Sunil Govind [mailto:sunil.gov...@gmail.com]
Sent: Wednesday, May 25, 2016 1:07 PM
To: Guttadauro, Jeff <jeff.guttada...@here.com>; user@hadoop.apache.org
Subject: Re: YARN cluster underutilization
Hi Jeff,
I
ldn’t
> you generally expect fairly stable utilization over the course of the job?
> (This is the only job running.)
>
>
>
> Thanks,
>
> -Jeff
>
>
>
> *From:* Sunil Govind [mailto:sunil.gov...@gmail.com]
> *Sent:* Wednesday, May 25, 2016 11:55 AM
>
>
> *To:* G
25, 2016 11:55 AM
To: Guttadauro, Jeff <jeff.guttada...@here.com>; user@hadoop.apache.org
Subject: Re: YARN cluster underutilization
Hi Jeff.
Thanks for sharing this information. I have some observations from this logs.
- I think the node heartbeat is around 2/3 seconds here. Is it chang
are pretty big, and I thought that might be sufficient.
>
>
>
> Any guidance is much appreciated!
>
> -Jeff
>
>
>
> *From:* Sunil Govind [mailto:sunil.gov...@gmail.com]
> *Sent:* Wednesday, May 25, 2016 10:55 AM
> *To:* Guttadauro, Jeff <jeff.guttada...@he
Hi Jeff,
It looks like to you are allocating more memory for AM container. Mostly
you might not need 6Gb (as per the log). Could you please help to provide
some more information.
1. What type of mapreduce application (wordcount etc) are you running? Some
AMs may be CPU intensive and some may
Hi, all.
I have an M/R (map-only) job that I'm running on a Hadoop 2.7.1 YARN cluster
that is being quite underutilized (utilization of around 25-30%). The EMR
cluster is 1 master + 20 core m3.xlarge nodes, which have 8 cores each and 15G
total memory (with 11.25G of that available to YARN).
14 matches
Mail list logo
