Hi Mahadev, Was this reproducible?
Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) >________________________________ > From: Andrew Purtell <apurt...@apache.org> >To: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org> >Cc: Mahadev Konar <maha...@hortonworks.com> >Sent: Wednesday, January 11, 2012 9:52 AM >Subject: Re: troubles with HBase unit tests using MiniMRCluster on >0.23.1-SNAPSHOT > > >Hi Mahadev, > > >> mvn -Dhadoop.profile=23 clean test >> -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce >> on hbase 0.92. (latest svn branch) should be able to recreate this? >> (Assuming your patch applied on MR-3654) > >Yes, but as Hitesh noted on MAPREDUCE-3654, it seems the Yarn client provider >is not found or used. (I don't know yet how all of this works...) So the issue >with LocalClientProvider not liking the MASTER_ADDRESS set to "test" by the >mini Yarn cluster is merely a symptom. Likewise perhaps the FNFE problem. > >> I can try that on my setup and see what the issue is. > >If you could spare a few minutes, that would be very helpful I think. Perhaps >our POM needs updating, some artifact is perhaps not being pulled in for testing, and while this was fine for 0.23.0, that is no longer the case for 0.23.1+? > > >Best regards, > > > - Andy > >Problems worthy of attack prove their worth by hitting back. - Piet Hein (via >Tom White) > > >>________________________________ >> From: Mahadev Konar <maha...@hortonworks.com> >>To: mapreduce-user@hadoop.apache.org; Andrew Purtell <apurt...@apache.org> >>Sent: Tuesday, January 10, 2012 6:30 PM >>Subject: Re: troubles with HBase unit tests using MiniMRCluster on >>0.23.1-SNAPSHOT >> >>Hi Andrew, >>Thanks for filing MAPREDUCE-3654. Ill take a look. As for the >>filenotfound issue >>mvn -Dhadoop.profile=23 clean test >>-Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce >> >>on hbase 0.92. (latest svn branch) should be able to recreate this? >>(Assuming your patch applied on MR-3654) >> >>I can try that on my setup and see what the issue is. >> >>thanks >>mahadev >> >>On Tue, Jan 10, 2012 at 6:11 PM, Andrew Purtell <apurt...@apache.org> wrote: >>> Hi, >>> >>> I've been trying to test HBase 0.92 (prerelease) with 0.23.1-SNAPSHOT but >>> have run into a couple of issues. Perhaps I'm doing something wrong. >>> >>> What I've done: >>> >>> - Checked out Hadoop branch-0.23 and installed artifacts into the local >>> maven cache: >>> >>> $ cd /path/to/hadoop >>> $ mvn clean install -DskipTests >>> >>> - Checked out HBase branch-0.92 and attempted to test under the maven >>> profile for 0.23: >>> >>> $ cd /path/to/hbase >>> $ mvn -Dhadoop.profile=23 clean test >>> -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce >>> >>> The first issue I encountered was that the MiniMRCluster would not >>> initialize: >>> >>> 2012-01-10 18:01:02,489 INFO [main] mapreduce.TestTableMapReduce(153): >>> Started mrtest >>> 2012-01-10 18:01:02,493 DEBUG [main] mapreduce.Cluster(93): Trying >>> ClientProtocolProvider : >>> org.apache.hadoop.mapred.LocalClientProtocolProvider >>> 2012-01-10 18:01:02,494 INFO [main] mapreduce.Cluster(116): Failed to use >>> org.apache.hadoop.mapred.LocalClientProtocolProvider due to error: Invalid >>> "mapreduce.jobtracker.address" configuration value for LocalJobRunner : >>> "test" >>> 2012-01-10 18:01:02,494 ERROR [main] security.UserGroupInformation(1160): >>> PriviledgedActionException as:apurtell (auth:SIMPLE) >>> cause:java.io.IOException: Cannot initialize Cluster. Please check your >>> configuration for mapreduce.framework.name and the correspond server >>> addresses. >>> 2012-01-10 18:01:02,497 INFO [main] hbase.HBaseTestingUtility(1173): >>> Stopping mini mapreduce cluster... >>> 2 >>> >>> I filed MAPREDUCE-3654 >>> (https://issues.apache.org/jira/browse/MAPREDUCE-3654). There could be more >>> going on here, but the patch on that issue let me move forward. >>> >>> Now I am seeing this: >>> >>> 2012-01-10 17:51:35,225 DEBUG [main] mapreduce.Cluster(106): Picked >>> org.apache.hadoop.mapred.LocalClientProtocolProvider as the >>> ClientProtocolProvider >>> 2012-01-10 17:51:35,243 INFO [main] mapreduce.TableOutputFormat(197): >>> Created table instance for mrtest >>> 2012-01-10 17:51:35,249 DEBUG [main] mapreduce.JobSubmitter(347): >>> Configuring job job_local_0001 with >>> file:/tmp/hadoop-apurtell/mapred/staging/apurtell-129099179/.staging/job_local_0001 >>> as the submit dir >>> 2012-01-10 17:51:35,251 DEBUG [main] mapreduce.JobSubmitter(565): adding the >>> following namenodes' delegation tokens:[hdfs://localhost.localdomain:54525] >>> 2012-01-10 17:51:35,251 WARN [main] mapreduce.JobSubmitter(139): Use >>> GenericOptionsParser for parsing the arguments. Applications should >>> implement Tool for the same. >>> 2012-01-10 17:51:35,252 DEBUG [main] mapreduce.JobSubmitter(158): default >>> FileSystem: file:/// >>> 2012-01-10 17:51:35,261 WARN [main] mapreduce.JobSubmitter(241): No job jar >>> file set. User classes may not be found. See Job or Job#setJar(String). >>> 2012-01-10 17:51:35,266 INFO [main] mapreduce.JobSubmitter(388): Cleaning >>> up the staging area >>> file:/tmp/hadoop-apurtell/mapred/staging/apurtell-129099179/.staging/job_local_0001 >>> 2012-01-10 17:51:35,268 ERROR [main] security.UserGroupInformation(1160): >>> PriviledgedActionException as:apurtell (auth:SIMPLE) >>> cause:java.io.FileNotFoundException: File does not exist: >>> /home/apurtell/.m2/repository/org/apache/zookeeper/zookeeper/3.4.2/zookeeper-3.4.2.jar >>> 2012-01-10 17:51:35,273 INFO [main] hbase.HBaseTestingUtility(1173): >>> Stopping mini mapreduce cluster... >>> >>> Of course that FileNotFoundException is for a file that does indeed exist, >>> in the local maven cache: >>> >>> $ ls -l >>> /home/apurtell/.m2/repository/org/apache/zookeeper/zookeeper/3.4.2/zookeeper-3.4.2.jar >>> -rw-rw-r-- 1 apurtell apurtell 764555 2012-01-05 18:56 >>> /home/apurtell/.m2/repository/org/apache/zookeeper/zookeeper/3.4.2/zookeeper-3.4.2.jar >>> >>> I set permissions on all directories from /home/apurtell to the jar to >>> 'rwxr-xr-x' just to make sure there was not some kind of odd permissions >>> problem. >>> >>> Any idea what is going on here? >>> >>> Best regards, >>> >>> - Andy >>> >>> >>> Problems worthy of attack prove their worth by hitting back. - Piet Hein >>> (via Tom White) >> >> >> > >
