Re: Yarn killing my Application Master

2013-10-12 Thread Jian He
By looking at the logs, it shows your application was killed and did not successfully registered with RM On Fri, Oct 11, 2013 at 3:53 PM, Pradeep Gollakota wrote: > All, > > I have a Yarn application that is launching a single container. The > container completes successfully but the applicatio

Mapreduce Combiner can't use the situation?

2013-10-12 Thread kun yan
Hi all Mapreduce Combiner can't use the situation? Now I know avg() is not suit -- In the Hadoop world, I am just a novice, explore the entire Hadoop ecosystem, I hope one day I can contribute their own code YanBit yankunhad...@gmail.com

Re: Hadoop Jobtracker heap size calculation and OOME

2013-10-12 Thread Viswanathan J
Hi Guys, But I can see the jobtracker OOME issue fixed in hadoop - 1.2.1 version as per the hadoop release notes as below. Please check this URL, https://issues.apache.org/jira/browse/MAPREDUCE-5351 How come the issue still persist? I'm I asking a valid thing. Do I need to configure anything o

Re: Writing to multiple directories in hadoop

2013-10-12 Thread Sonal Goyal
Hi Jamal, If I remember correctly, you can use the write(key, value, basePath) method of MultipleOutput in your reducer to get different directories. http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html#write(KEYOUT, VALUEOUT, java.lang.String) H

Re: Hadoop Jobtracker heap size calculation and OOME

2013-10-12 Thread Viswanathan J
Thanks Antonio, hope the memory leak issue will be resolved. Its really nightmare every week. In which release this issue will be resolved? How to solve this issue, please help because we are facing in production environment. Please share the configuration and cron to do that cleanup process. T

Re: Hadoop Jobtracker heap size calculation and OOME

2013-10-12 Thread Viswanathan J
Hi Harsh, Appreciate the response. Thanks Reyane. Thanks, Viswa.J On Oct 12, 2013 5:04 AM, "Reyane Oukpedjo" wrote: > Hi there, > I had a similar issue with hadoop-1.2.0 JobTracker keep crashing until I > set HADOOP_HEAPSIZE="2048" I did not have this kind of issue with > previous versions.

./bin/hdfs haadmin -transitionToActive deadlock

2013-10-12 Thread lei liu
I use CDH4.3.1, When I start NameNode,and transition one NameNode to active, there is below deadlock: Found one Java-level deadlock: = "22558696@qtp-1616586953-6": waiting to lock monitor 0x2aaab3621f40 (object 0xf7646958, a org.apache.hadoop.hdfs.server.n