Yeah that was my first thought. rowcounter and PerformanceEvaluation work. here is my HADOOP_CLASSPATH: =================================== export HADOOP_CLASSPATH=$HBASE_HOME/hbase-0.19.3.jar:$HBASE_HOME/hbase-0.19.3-test.jar:/sharedfs/hbase-config/conf =================================== slave boxes reference a shared conf: export HADOOP_CLASSPATH=$HBASE_HOME/hbase-0.19.3.jar:$HBASE_HOME/hbase-0.19.3-test.jar:$HBASE_HOME/conf =================================== where HBASE_HOME=/home/hadoop/hbase193. this is the same path on all boxes.
I run my .class path from the default package like so: hadoop19/bin/hadoop ColumnCountMapReduce inputTableName stack-3 wrote: > > Thats a CLASSPATH issue. For sure the hbase jars are out on all nodes of > your cluster? For sure the paths are correct? > St.Ack > > On Tue, Jun 9, 2009 at 10:28 AM, llpind <[email protected]> wrote: > >> >> Yep. Thanks, I didn't think to look there :). I do have another issue >> when >> running my own map/reduce. I've got the jar files in HADOOP_CLASSPATH as >> i >> mentioned above, but when i run the job below is what i get. The >> ClassDefNotFoundError happens on TableSplit multiple times. >> ================================================ >> >> hadoop19/bin/hadoop ColumnCountMapReduce inputTableName >> 09/06/09 10:26:08 WARN mapred.JobClient: No job jar file set. User >> classes >> may not be found. See JobConf(Class) or JobConf#setJar(String). >> 09/06/09 10:26:11 INFO mapred.JobClient: Running job: >> job_200906090918_0005 >> 09/06/09 10:26:12 INFO mapred.JobClient: map 0% reduce 0% >> 09/06/09 10:26:21 INFO mapred.JobClient: Task Id : >> attempt_200906090918_0005_m_000000_0, Status : FAILED >> java.io.IOException: Split class >> org.apache.hadoop.hbase.mapred.TableSplit >> not found >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314) >> at org.apache.hadoop.mapred.Child.main(Child.java:158) >> Caused by: java.lang.ClassNotFoundException: >> org.apache.hadoop.hbase.mapred.TableSplit >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:252) >> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:247) >> at >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311) >> ... 1 more >> >> 09/06/09 10:26:21 INFO mapred.JobClient: Task Id : >> attempt_200906090918_0005_m_000004_0, Status : FAILED >> java.io.IOException: Split class >> org.apache.hadoop.hbase.mapred.TableSplit >> not found >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314) >> at org.apache.hadoop.mapred.Child.main(Child.java:158) >> Caused by: java.lang.ClassNotFoundException: >> org.apache.hadoop.hbase.mapred.TableSplit >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:252) >> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:247) >> at >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311) >> ... 1 more >> >> >> stack-3 wrote: >> > >> > It looks pretty obvious what your issue is. >> > >> > My guess is that your child process needs its heap size bumped up. >> > >> > Look for the child parameters configuration in your hadoop-default.xml >> (or >> > your mapred-default.xml if 0.20.x hadoop). >> > >> > St.Ack >> > >> > On Tue, Jun 9, 2009 at 10:05 AM, llpind <[email protected]> wrote: >> > >> >> >> >> Hmm...jobtracker UI has the following: >> >> >> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to >> >> contact >> >> region server 192.168.0.195:60020 for region TestTable,,1244499094604, >> >> row >> >> '0000010477', but failed after 10 attempts. >> >> Exceptions: >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> >> >> at >> >> >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:859) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:951) >> >> at >> >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1397) >> >> at >> org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1341) >> >> at >> org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1321) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest.testRow(PerformanceEvaluation.java:515) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:386) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:583) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:182) >> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) >> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) >> >> at org.apache.hadoop.mapred.Child.main(Child.java:158) >> >> >> >> ================================================= >> >> >> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to >> >> contact >> >> region server 192.168.0.195:60020 for region TestTable,,1244499094604, >> >> row >> >> '0002107629', but failed after 10 attempts. >> >> Exceptions: >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> java.lang.OutOfMemoryError: Java heap space >> >> >> >> at >> >> >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:859) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:951) >> >> at >> >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1397) >> >> at >> org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1341) >> >> at >> org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1321) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest.testRow(PerformanceEvaluation.java:515) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:386) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:583) >> >> at >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:182) >> >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) >> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) >> >> at org.apache.hadoop.mapred.Child.main(Child.java:158) >> >> >> >> >> >> >> >> >> >> stack-3 wrote: >> >> > >> >> > Look at why the job failed in the jobtracker UI -- usually on port >> >> 50030. >> >> > Looks like your job launched fine. You have the conf dir on the >> >> > HADOOP_CLASSPATH so the MR job can find hbase? >> >> > St.Ack >> >> > >> >> > On Tue, Jun 9, 2009 at 9:25 AM, llpind <[email protected]> >> wrote: >> >> > >> >> >> >> >> >> Hey all, >> >> >> >> >> >> getting started with map/reduce jobs. Figured I'd try the >> >> >> PerformanceEvaluation test program first. >> >> >> >> >> >> After adding HBase jar files to HADOOP_CLASSPATH in hadoop-env.sh, >> I >> >> >> issue >> >> >> the following command: >> >> >> >> >> >> hadoop19/bin/hadoop org.apache.hadoop.hbase.PerformanceEvaluation >> >> >> sequentialWrite 4 >> >> >> >> >> >> and get the following output: >> >> >> >> >> >> 09/06/09 09:20:54 WARN mapred.JobClient: Use GenericOptionsParser >> for >> >> >> parsing the arguments. Applications should implement Tool for the >> >> same. >> >> >> 09/06/09 09:20:55 INFO mapred.FileInputFormat: Total input paths to >> >> >> process >> >> >> : 1 >> >> >> 09/06/09 09:20:56 INFO mapred.JobClient: Running job: >> >> >> job_200906090918_0001 >> >> >> 09/06/09 09:20:57 INFO mapred.JobClient: map 0% reduce 0% >> >> >> java.io.IOException: Job failed! >> >> >> at >> >> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation.doMapReduce(PerformanceEvaluation.java:293) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation.runNIsMoreThanOne(PerformanceEvaluation.java:221) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation.runTest(PerformanceEvaluation.java:639) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation.doCommandLine(PerformanceEvaluation.java:748) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:768) >> >> >> >> >> >> >> >> >> What am I doing wrong...? >> >> >> >> >> >> >> >> >> -- >> >> >> View this message in context: >> >> >> >> >> >> http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23946437.html >> >> >> Sent from the HBase User mailing list archive at Nabble.com. >> >> >> >> >> >> >> >> > >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23947225.html >> >> Sent from the HBase User mailing list archive at Nabble.com. >> >> >> >> >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23947629.html >> Sent from the HBase User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23947980.html Sent from the HBase User mailing list archive at Nabble.com.
