Thats a CLASSPATH issue. For sure the hbase jars are out on all nodes of your cluster? For sure the paths are correct? St.Ack
On Tue, Jun 9, 2009 at 10:28 AM, llpind <[email protected]> wrote: > > Yep. Thanks, I didn't think to look there :). I do have another issue > when > running my own map/reduce. I've got the jar files in HADOOP_CLASSPATH as i > mentioned above, but when i run the job below is what i get. The > ClassDefNotFoundError happens on TableSplit multiple times. > ================================================ > > hadoop19/bin/hadoop ColumnCountMapReduce inputTableName > 09/06/09 10:26:08 WARN mapred.JobClient: No job jar file set. User classes > may not be found. See JobConf(Class) or JobConf#setJar(String). > 09/06/09 10:26:11 INFO mapred.JobClient: Running job: job_200906090918_0005 > 09/06/09 10:26:12 INFO mapred.JobClient: map 0% reduce 0% > 09/06/09 10:26:21 INFO mapred.JobClient: Task Id : > attempt_200906090918_0005_m_000000_0, Status : FAILED > java.io.IOException: Split class org.apache.hadoop.hbase.mapred.TableSplit > not found > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314) > at org.apache.hadoop.mapred.Child.main(Child.java:158) > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.mapred.TableSplit > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:252) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:247) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311) > ... 1 more > > 09/06/09 10:26:21 INFO mapred.JobClient: Task Id : > attempt_200906090918_0005_m_000004_0, Status : FAILED > java.io.IOException: Split class org.apache.hadoop.hbase.mapred.TableSplit > not found > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314) > at org.apache.hadoop.mapred.Child.main(Child.java:158) > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hbase.mapred.TableSplit > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) > at java.lang.ClassLoader.loadClass(ClassLoader.java:252) > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:247) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311) > ... 1 more > > > stack-3 wrote: > > > > It looks pretty obvious what your issue is. > > > > My guess is that your child process needs its heap size bumped up. > > > > Look for the child parameters configuration in your hadoop-default.xml > (or > > your mapred-default.xml if 0.20.x hadoop). > > > > St.Ack > > > > On Tue, Jun 9, 2009 at 10:05 AM, llpind <[email protected]> wrote: > > > >> > >> Hmm...jobtracker UI has the following: > >> > >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to > >> contact > >> region server 192.168.0.195:60020 for region TestTable,,1244499094604, > >> row > >> '0000010477', but failed after 10 attempts. > >> Exceptions: > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:859) > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:951) > >> at > >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1397) > >> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1341) > >> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1321) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest.testRow(PerformanceEvaluation.java:515) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:386) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:583) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:182) > >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > >> at org.apache.hadoop.mapred.Child.main(Child.java:158) > >> > >> ================================================= > >> > >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to > >> contact > >> region server 192.168.0.195:60020 for region TestTable,,1244499094604, > >> row > >> '0002107629', but failed after 10 attempts. > >> Exceptions: > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> java.lang.OutOfMemoryError: Java heap space > >> > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:859) > >> at > >> > >> > org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:951) > >> at > >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1397) > >> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1341) > >> at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1321) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest.testRow(PerformanceEvaluation.java:515) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:386) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:583) > >> at > >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:182) > >> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > >> at org.apache.hadoop.mapred.Child.main(Child.java:158) > >> > >> > >> > >> > >> stack-3 wrote: > >> > > >> > Look at why the job failed in the jobtracker UI -- usually on port > >> 50030. > >> > Looks like your job launched fine. You have the conf dir on the > >> > HADOOP_CLASSPATH so the MR job can find hbase? > >> > St.Ack > >> > > >> > On Tue, Jun 9, 2009 at 9:25 AM, llpind <[email protected]> > wrote: > >> > > >> >> > >> >> Hey all, > >> >> > >> >> getting started with map/reduce jobs. Figured I'd try the > >> >> PerformanceEvaluation test program first. > >> >> > >> >> After adding HBase jar files to HADOOP_CLASSPATH in hadoop-env.sh, I > >> >> issue > >> >> the following command: > >> >> > >> >> hadoop19/bin/hadoop org.apache.hadoop.hbase.PerformanceEvaluation > >> >> sequentialWrite 4 > >> >> > >> >> and get the following output: > >> >> > >> >> 09/06/09 09:20:54 WARN mapred.JobClient: Use GenericOptionsParser for > >> >> parsing the arguments. Applications should implement Tool for the > >> same. > >> >> 09/06/09 09:20:55 INFO mapred.FileInputFormat: Total input paths to > >> >> process > >> >> : 1 > >> >> 09/06/09 09:20:56 INFO mapred.JobClient: Running job: > >> >> job_200906090918_0001 > >> >> 09/06/09 09:20:57 INFO mapred.JobClient: map 0% reduce 0% > >> >> java.io.IOException: Job failed! > >> >> at > >> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232) > >> >> at > >> >> > >> >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation.doMapReduce(PerformanceEvaluation.java:293) > >> >> at > >> >> > >> >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation.runNIsMoreThanOne(PerformanceEvaluation.java:221) > >> >> at > >> >> > >> >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation.runTest(PerformanceEvaluation.java:639) > >> >> at > >> >> > >> >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation.doCommandLine(PerformanceEvaluation.java:748) > >> >> at > >> >> > >> >> > >> > org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:768) > >> >> > >> >> > >> >> What am I doing wrong...? > >> >> > >> >> > >> >> -- > >> >> View this message in context: > >> >> > >> > http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23946437.html > >> >> Sent from the HBase User mailing list archive at Nabble.com. > >> >> > >> >> > >> > > >> > > >> > >> -- > >> View this message in context: > >> > http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23947225.html > >> Sent from the HBase User mailing list archive at Nabble.com. > >> > >> > > > > > > -- > View this message in context: > http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23947629.html > Sent from the HBase User mailing list archive at Nabble.com. > >
