Re: Intermittent BindException during long MR jobs

2015-02-28 Thread Ted Yu
Krishna: Please take a look at: http://wiki.apache.org/hadoop/BindException Cheers On Thu, Feb 26, 2015 at 10:30 PM, hadoop.supp...@visolve.com wrote: Hello Krishna, Exception seems to be IP specific. It might be occurred due to unavailability of IP address in the system to assign. Double

Re: cleanup() in hadoop results in aggregation of whole file/not

2015-02-28 Thread Shahab Yunus
As far as I understand cleanup is called per task. In your case I.e. per map task. To get an overall count or measure, you need to aggregate it yourself after the job is done. One way to do that is to use counters and then merge them programmatically at the end of the job. Regards, Shahab On

Re: unable to build oozie

2015-02-28 Thread Ted Yu
Looks like this question should be directed to oozie mailing list: http://oozie.apache.org/mail-lists.html Cheers On Sat, Feb 28, 2015 at 12:20 PM, hitarth trivedi t.hita...@gmail.com wrote: Hi, I downloaded the latest release oozie-4.0.1 . When I try to build it locally using

RE: Hadoop 2.6.0 - No DataNode to stop

2015-02-28 Thread Daniel Klinger
Thanks but i know how to kill a process in Linux. But this didn’t answer the question why the command say no Datanode to stop instead of stopping the Datanode: $HADOOP_PREFIX/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR --script hdfs stop datanode Von: Surbhi Gupta

Re: Hadoop 2.6.0 - No DataNode to stop

2015-02-28 Thread Surbhi Gupta
Issue jps and get the process id or Try to get the process id of datanode. Issue ps-fu userid of the user through which datanode is running. Then kill the process using kill -9 On 28 Feb 2015 09:38, Daniel Klinger d...@web-computing.de wrote: Hello, I used a lot of Hadoop-Distributions.

unable to build oozie

2015-02-28 Thread hitarth trivedi
Hi, I downloaded the latest release oozie-4.0.1 . When I try to build it locally using bin/mkdistro.sh I get following error, Error resolving version for plugin 'com.atlassian.maven.plugins:maven-clover2-plugin' from the repositories [local (/home/cmx/.m2/repository), repository.cloudera.com (

Hadoop 2.6.0 - No DataNode to stop

2015-02-28 Thread Daniel Klinger
Hello, I used a lot of Hadoop-Distributions. Now I'm trying to install a pure Hadoop on a little cluster for testing (2 CentOS-VMs: 1 Name+DataNode 1 DataNode). I followed the instructions on the Documentation site: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Clus

Running Highly Available KMS

2015-02-28 Thread Rahul Singhal
Hi All, I am trying to run a highly available KMS service using two instances of KMS (using 2.6.0). But I am having problem initializing the keystore on HDFS. I think the problem is that hdfs-site.xml is not being loaded by KMS. Has anybody else tried to do this? Are there any other