DisallowedDatanodeException: Datanode denied communication with namenode

I am trying to launch the datanodes in Hadoop MRv2, and I get the error below. I looked to Hadoop conf files and the /etc/hosts and everything looks ok. What is wrong in my configuration? org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with nameno

restrict job queues listing

Hi, I am trying to secure multiple MR job queues from being accessed from unauthorized users. I know there is "acl_submit_applications" and "acl_administer_queue" for a specific queue, but neither of those properties control who can view the list of jobs that have been or are executing in a speci

java.lang.OutOfMemoryError: Java heap space

Hi all, I am running a Pig Script which is running fine for small data. But when I scale the data, I am getting the following error at my map stage. Please refer to the map logs as below. My Pig script is doing a group by first, followed by a join on the grouped data. Any clues to understand wh

Re: java.lang.OutOfMemoryError: Java heap space

Its a normal join. I can't use replicated join, as the data is very large. Regards Prav On Thu, Feb 6, 2014 at 7:52 PM, abhishek wrote: > Hi Praveenesh, > > Did you use "replicated join" in your pig script or is it a regular join ?? > > Regards > Abhishek > > Sent from my iPhone > > > On Feb 6

Problems building hadoop 2.2.0 from source

Hi, I am a complete beginner to Hadoop, trying to build 2.2.0 from source on a Macbook Pro running OS X Mavericks. I am following the 'instructions' at http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html such as they are. I get the following test failure:

Re: Problems building hadoop 2.2.0 from source

There isn't System.exit call in TestMRJobsWithHistoryService.java What did hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt say ? Cheers On Thu, Feb 6, 2014 at 4:41 PM, Chr

Re: Problems building hadoop 2.2.0 from source

I checked out source code from http://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.3 and it builds. >From TestMRJobsWithHistoryService.txt, the test passed. What led to this test being singled out among the 454 tests ? Thanks On Thu, Feb 6, 2014 at 7:26 PM, Christopher Thomas < chri

Re: Problems building hadoop 2.2.0 from source

Yes well I tried 2.3, but I have found a number of problems building it. I had to resort to manually applying patches that I found in the bug tracking lists, which did not seem to have made it into all branches. So for the moment I am sticking with 2.2.0 which is advertised as being stable. I apol

Re: Problems building hadoop 2.2.0 from source

The output was from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-output.txt Can you show us the contents of ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduc

Re: Problems building hadoop 2.2.0 from source

I included the last part of hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client- jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2. TestMRJobsWithHistoryService.txt in the second half of my initial posting, calling it "the output from TestMRJobsWithHistoryServi

Re: Problems building hadoop 2.2.0 from source

I guess the ERROR lines in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client- jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2. TestMRJobsWithHistoryService-output.txt led me to believe that the problem was with TestMRJobsWithHistoryService. If that's not th