Hi, my job config is
<property>
<name>mapred.input.format.class</name>
<value>org.apache.hadoop.hbase.mapred.TableInputFormat</value>
</property>
<property>
<name>hadoop.pipes.java.recordreader</name>
<value>true</value>
</property>
Exception in thread "main" java.lang.RuntimeException: Error in
configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:596)
at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:977)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:969)
at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1248)
at org.apache.hadoop.mapred.pipes.Submitter.runJob(Submitter.java:248)
at org.apache.hadoop.mapred.pipes.Submitter.run(Submitter.java:479)
at org.apache.hadoop.mapred.pipes.Submitter.main(Submitter.java:494)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.mapred.TableInputFormat.configure(TableInputFormat.java:51)
should I included the col names? according to the api it's deprecated?
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapred/TableInputFormat.html
-Håvard
On Tue, Aug 14, 2012 at 11:17 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> Per:
>
>> org.apache.hadoop.hbase.mapreduce.TableInputFormat not
> org.apache.hadoop.mapred.InputFormat
>
> Pydoop seems to be expecting you to pass it an old API class for
> InputFormat/etc. but you've passed in the newer class. I am unsure
> what part of your code exactly may be at fault since I do not have
> access to it, but you probably want to use the deprecated
> org.apache.hadoop.hbase.mapred.* package classes such as
> org.apache.hadoop.hbase.mapred.TableInputFormat, and not the
> org.apache.hadoop.hbase.mapreduce.* classes, as you are using at the
> moment.
>
> HTH!
>
> On Wed, Aug 15, 2012 at 2:39 AM, Håvard Wahl Kongsgård
> <haavard.kongsga...@gmail.com> wrote:
>> Hi, I'am trying to read hbase key-values with pipes(pydoop). As hadoop
>> is unable to find the hbase jar files. I get
>>
>> Exception in thread "main" java.lang.RuntimeException:
>> java.lang.RuntimeException: class
>> org.apache.hadoop.hbase.mapreduce.TableInputFormat not
>> org.apache.hadoop.mapred.InputFormat
>>
>> have added export
>> HADOOP_CLASSPATH=/usr/lib/hbase/hbase-0.90.6-cdh3u4.jar to my
>> hadoop-env.sh
>>
>> According to the doc from cloudera,
>> https://ccp.cloudera.com/display/CDHDOC/HBase+Installation#HBaseInstallation-UsingMapReducewithHBase
>> TableMapReduceUtil.addDependencyJars(job); can be used as an
>> alternative. But is that possible with pipes?
>>
>> -Håvard
>
>
>
> --
> Harsh J
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
