Thats a CLASSPATH issue.  For sure the hbase jars are out on all nodes of
your cluster?  For sure the paths are correct?
St.Ack

On Tue, Jun 9, 2009 at 10:28 AM, llpind <[email protected]> wrote:

>
> Yep.  Thanks,  I didn't think to look there :).  I do have another issue
> when
> running my own map/reduce.  I've got the jar files in HADOOP_CLASSPATH as i
> mentioned above, but when i run the job below is what i get.  The
> ClassDefNotFoundError happens on TableSplit multiple times.
> ================================================
>
> hadoop19/bin/hadoop ColumnCountMapReduce inputTableName
> 09/06/09 10:26:08 WARN mapred.JobClient: No job jar file set.  User classes
> may not be found. See JobConf(Class) or JobConf#setJar(String).
> 09/06/09 10:26:11 INFO mapred.JobClient: Running job: job_200906090918_0005
> 09/06/09 10:26:12 INFO mapred.JobClient:  map 0% reduce 0%
> 09/06/09 10:26:21 INFO mapred.JobClient: Task Id :
> attempt_200906090918_0005_m_000000_0, Status : FAILED
> java.io.IOException: Split class org.apache.hadoop.hbase.mapred.TableSplit
> not found
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314)
>         at org.apache.hadoop.mapred.Child.main(Child.java:158)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapred.TableSplit
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
>        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
>        at java.lang.Class.forName0(Native Method)
>        at java.lang.Class.forName(Class.java:247)
>        at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311)
>        ... 1 more
>
> 09/06/09 10:26:21 INFO mapred.JobClient: Task Id :
> attempt_200906090918_0005_m_000004_0, Status : FAILED
> java.io.IOException: Split class org.apache.hadoop.hbase.mapred.TableSplit
> not found
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314)
>         at org.apache.hadoop.mapred.Child.main(Child.java:158)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapred.TableSplit
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
>        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
>        at java.lang.Class.forName0(Native Method)
>        at java.lang.Class.forName(Class.java:247)
>        at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311)
>        ... 1 more
>
>
> stack-3 wrote:
> >
> > It looks pretty obvious what your issue is.
> >
> > My guess is that your child process needs its heap size bumped up.
> >
> > Look for the child parameters configuration in your hadoop-default.xml
> (or
> > your mapred-default.xml if 0.20.x hadoop).
> >
> > St.Ack
> >
> > On Tue, Jun 9, 2009 at 10:05 AM, llpind <[email protected]> wrote:
> >
> >>
> >> Hmm...jobtracker UI has the following:
> >>
> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> >> contact
> >> region server 192.168.0.195:60020 for region TestTable,,1244499094604,
> >> row
> >> '0000010477', but failed after 10 attempts.
> >> Exceptions:
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >>
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:859)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:951)
> >>        at
> >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1397)
> >>        at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1341)
> >>        at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1321)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest.testRow(PerformanceEvaluation.java:515)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:386)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:583)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:182)
> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> >>        at org.apache.hadoop.mapred.Child.main(Child.java:158)
> >>
> >> =================================================
> >>
> >> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> >> contact
> >> region server 192.168.0.195:60020 for region TestTable,,1244499094604,
> >> row
> >> '0002107629', but failed after 10 attempts.
> >> Exceptions:
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >> java.lang.OutOfMemoryError: Java heap space
> >>
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:859)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:951)
> >>        at
> >> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1397)
> >>        at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1341)
> >>        at org.apache.hadoop.hbase.client.HTable.commit(HTable.java:1321)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest.testRow(PerformanceEvaluation.java:515)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:386)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:583)
> >>        at
> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:182)
> >>        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> >>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
> >>        at org.apache.hadoop.mapred.Child.main(Child.java:158)
> >>
> >>
> >>
> >>
> >> stack-3 wrote:
> >> >
> >> > Look at why the job failed in the jobtracker UI -- usually on port
> >> 50030.
> >> > Looks like your job launched fine.   You have the conf dir on the
> >> > HADOOP_CLASSPATH so the MR job can find hbase?
> >> > St.Ack
> >> >
> >> > On Tue, Jun 9, 2009 at 9:25 AM, llpind <[email protected]>
> wrote:
> >> >
> >> >>
> >> >> Hey all,
> >> >>
> >> >> getting started with map/reduce jobs.  Figured I'd try the
> >> >> PerformanceEvaluation test program first.
> >> >>
> >> >> After adding HBase jar files to HADOOP_CLASSPATH in hadoop-env.sh, I
> >> >> issue
> >> >> the following command:
> >> >>
> >> >> hadoop19/bin/hadoop org.apache.hadoop.hbase.PerformanceEvaluation
> >> >> sequentialWrite 4
> >> >>
> >> >> and get the following output:
> >> >>
> >> >> 09/06/09 09:20:54 WARN mapred.JobClient: Use GenericOptionsParser for
> >> >> parsing the arguments. Applications should implement Tool for the
> >> same.
> >> >> 09/06/09 09:20:55 INFO mapred.FileInputFormat: Total input paths to
> >> >> process
> >> >> : 1
> >> >> 09/06/09 09:20:56 INFO mapred.JobClient: Running job:
> >> >> job_200906090918_0001
> >> >> 09/06/09 09:20:57 INFO mapred.JobClient:  map 0% reduce 0%
> >> >> java.io.IOException: Job failed!
> >> >>        at
> >> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
> >> >>        at
> >> >>
> >> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation.doMapReduce(PerformanceEvaluation.java:293)
> >> >>        at
> >> >>
> >> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation.runNIsMoreThanOne(PerformanceEvaluation.java:221)
> >> >>        at
> >> >>
> >> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation.runTest(PerformanceEvaluation.java:639)
> >> >>        at
> >> >>
> >> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation.doCommandLine(PerformanceEvaluation.java:748)
> >> >>        at
> >> >>
> >> >>
> >>
> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:768)
> >> >>
> >> >>
> >> >> What am I doing wrong...?
> >> >>
> >> >>
> >> >> --
> >> >> View this message in context:
> >> >>
> >>
> http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23946437.html
> >> >> Sent from the HBase User mailing list archive at Nabble.com.
> >> >>
> >> >>
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23947225.html
> >> Sent from the HBase User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/PerformanceEvaluation-test-tp23946437p23947629.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>

Reply via email to