Starting YARN in HA mode Hadoop 2.5.1

Hi, I am using Hadoop 2.5.1 and trying to enable HA mode for NN and RM. the start-dfs.sh script provided starts both the namenodes and all the datanodes. However, the start-yarn.sh starts only one RM and all the NodeManagers. till now, these scripts allowed me to startup the entire cluster from

Re: Fwd: Values getting duplicated in Hive table(Partitioned)

Thanks it worked. On Nov 17, 2014 3:32 PM, unmesha sreeveni unmeshab...@gmail.com wrote: -- Forwarded message -- From: unmesha sreeveni unmeshab...@gmail.com Date: Mon, Nov 17, 2014 at 10:49 AM Subject: Re: Values getting duplicated in Hive table(Partitioned) To: User - Hive

MR job fails with too many mappers

Hi All, I have a small hadoop cluster with three nodes and HBase 0.98.1 installed on it. The hadoop version is 2.3.0 and below my use case scenario. I wrote a map reduce program that reads data from an hbase table and does some transformations on these data. Jobs are very simple so they didn't

RE: MR job fails with too many mappers

Hi Could you get syserr and sysout log for contrainer.? These logs will be available in the same location syslog for container. ${yarn.nodemanager.log-dirs}/app-id/container-id This helps to find problem!! Thanks Regards Rohith Sharma K S From: francexo83 [mailto:francex...@gmail.com] Sent:

Re: MR job fails with too many mappers

Hi, Could you share following configurations? It can be failures because of out of memory at mapper side. yarn.app.mapreduce.am.resource.mb mapreduce.map.memory.mb mapreduce.reduce.memory.mb mapreduce.map.java.opts mapreduce.reduce.java.opts On Wed, Nov 19, 2014 at 12:23 AM, francexo83

RE: Starting YARN in HA mode Hadoop 2.5.1

You need to start manually. Yarn does not support starting all the RM’s in cluster. Thanks Regards Rohith Sharma K S From: Jogeshwar Karthik Akundi [mailto:ajkart...@gmail.com] Sent: 18 November 2014 19:15 To: user@hadoop.apache.org Subject: Starting YARN in HA mode Hadoop 2.5.1 Hi, I am

Re: MR job fails with too many mappers

Hi, thank you for your quick response, but I was not able to see the logs for the container. I get a no such file or directory when I try to access the logs of the container from the shell: cd /var/log/hadoop-yarn/containers/application_1416304409718_0032 It seems that the container has

Re: MR job fails with too many mappers

Hi Tsuyoshi, these are the configurations you requested: yarn.app.mapreduce.am.resource.mb=256 mapreduce.map.memory.mb=Not set mapreduce.reduce.memory.mb=Not set mapreduce.map.java.opts=Not set mapreduce.reduce.java.opts=Not set thanks 2014-11-18 17:01 GMT+01:00 Tsuyoshi OZAWA

RE: MR job fails with too many mappers

If log aggregation is enabled, log folder will be deleted. So I suggest disable “yarn.log-aggregation-enable” and run job again. All the logs remains at log folder. Then you can find container logs Thanks Regards Rohith Sharma K S This e-mail and its attachments contain confidential

How does the project resolve conflicts between members?

As subject, thanks!

[Blog] Hive Partitioning

Hi, This is a blog on Hive partitioning. http://unmeshasreeveni.blogspot.in/2014/11/hive-partitioning.html Hope it helps someone. -- *Thanks Regards * *Unmesha Sreeveni U.B* *Hadoop, Bigdata Developer* *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*

[Blog] Updating Partition Table using INSERT In HIVE

Hi This is a blog on Hive updating for older version (hive -0.12.0) http://unmeshasreeveni.blogspot.in/2014/11/updating-partition-table-using-insert.html Hope it helps someone. -- *Thanks Regards * *Unmesha Sreeveni U.B* *Hadoop, Bigdata Developer* *Centre for Cyber Security | Amrita

Re: About HDFS-RAID

Thanks Liu But I found that the HDFS-7285 still open, so it means that there are no patch for the Hadoop 2.2.0 use by now, i am right ? 2014-11-18 14:48 GMT+08:00 Liu, Yi A yi.a@intel.com: HDFS-RAID is not maintained anymore and has been removed from Hadoop. Now we are re-design Erasure