What is the minimum container size? i.e. yarn.scheduler.minimum-allocation-mb.
I'd bump it up to at least 1G and use the CapacityScheduler for performance
tests:
http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
In case of teragen, the job has
I ran into the same issue. In the end I gave up and went back to 0.20 where I
can specify the number of mappers and reducers per node (6 and 4 in your case).
You can try increasing the memory.mb parameters which should force fewer
map/reduce tasks per node, but then you won't be able to run yo
Hello,
I'm trying to tune terasort on a small cluster (4 identical slave
nodes w/ 4 disks and 16GB RAM each), but I'm having problems with very
uneven load.
For teragen, I specify 24 mappers, but for some reason, only 2 nodes
out of 4 run them all, even though the web UI (for both YARN and HDFS)
Task tracker doesn't clean any data on restart. What is the data you are
storing(job oputput or something else)? Can you check the task tracker logs or
any other process removing which is running in the same machine.
Thanks
Devaraj
From: hadoop anis [had
Thanks for Replying,
I am using shared directory to store the data
On 5/29/12, Devaraj k wrote:
> What is the local directory you are using to store the data?
>
> Thanks
> Devaraj
>
>
> From: hadoop anis [hadoop.a...@gmail.com]
> Sent: Tuesday,
What is the local directory you are using to store the data?
Thanks
Devaraj
From: hadoop anis [hadoop.a...@gmail.com]
Sent: Tuesday, May 29, 2012 12:29 PM
To: mapreduce-user@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Subject: Re: cleanup of data wh
Hi friends,
I am using Hadoop 0.20.2 version.
>
>In my setup When Tasktracker exits, then data persist on
> linux filesystem. (I am using Hadoop without HDFS)
> but when I restart the tasktracker on that node it cleans all data on
> it's directory.
>
>
