Re: hbase bulk load with map reduce error

2016-06-15 Thread Jilani Shaik 
Please let me know what am I missing here.

I am using MapR hadoop. Please find the classpath which is showing all the
jars with versions, mean while I will get the code snippet also. I am using
bulk write approach with mapper and reducer. HBase table created with bulk
load enabled as true.

+ HADOOP_CLASSPATH='/opt/mapr/hbase/hbase-0.98.9/bin/../
conf:/opt/mapr/java/jdk1.7.0_25/lib/tools.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/..:/opt/mapr/lib/zookeeper-3.4.5-mapr-1406.jar:/opt/mapr/hbase/
hbase-0.98.9/bin/../lib/activation-1.1.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/aopalliance-1.0.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/api-asn1-api-1.0.0-M20.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/api-util-1.0.0-M20.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/asm-3.1.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/commons-beanutils-1.7.0.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/commons-cli-1.2.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/commons-codec-1.7.jar:/opt/mapr/hbase/hbase-
0.98.9/bin/../lib/commons-collections-3.2.1.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/commons-compress-1.4.1.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/commons-configuration-1.6.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/commons-daemon-1.0.13.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/commons-digester-1.8.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/commons-el-1.0.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/commons-httpclient-3.1.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/commons-io-2.4.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/commons-lang-2.6.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/commons-logging-1.1.1.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/commons-math-2.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
/lib/commons-math3-3.1.1.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/commons-net-3.1.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/findbugs-annotations-1.3.9-1.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/guava-12.0.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
/lib/guice-3.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
guice-servlet-3.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
/lib/hamcrest-core-1.3.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase
-annotations-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
hbase-checkstyle-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/hbase-client-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/hbase-common-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/hbase-common-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/
hbase-0.98.9/bin/../lib/hbase-examples-0.98.9-mapr-1503.jar:/opt/mapr/hbase/
hbase-0.98.9/bin/../lib/hbase-hadoop2-compat-0.98.9-mapr-1503.jar:/opt/mapr/
hbase/hbase-0.98.9/bin/../lib/hbase-hadoop-compat-0.98.9-mapr-1503.jar:/
opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase-it-0.98.9-
mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase-
it-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase
-prefix-tree-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
hbase-protocol-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
hbase-rest-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
hbase-server-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/
hbase-server-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/hbase-shell-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/hbase-testing-util-0.98.9-mapr-1503.jar:/opt/mapr/hbase/
hbase-0.98.9/bin/../lib/hbase-thrift-0.98.9-mapr-1503.jar:/opt/mapr/hbase/
hbase-0.98.9/bin/../lib/high-scale-lib-1.1.1.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/htrace-core-2.04.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
/lib/httpclient-4.2.5.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/httpcore-4.1.3.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/jackson-core-asl-1.8.8.jar:/opt/mapr/hbase/hbase-
0.98.9/bin/../lib/jackson-jaxrs-1.8.8.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/jackson-mapper-asl-1.8.8.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/jackson-xc-1.8.8.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/jamon-runtime-2.3.1.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/jasper-compiler-5.5.23.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/jasper-runtime-5.5.23.jar:/opt/mapr/hbase/hbase-
0.98.9/bin/../lib/javax.inject-1.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/java-xmlbuilder-0.4.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/jaxb-api-2.2.2.jar:/opt/mapr/hbase/hbase
-0.98.9/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
/lib/jcodings-1.0.8.jar:/opt/mapr/hbase/hbase-0.98.9/bin/..
/lib/jersey-client-1.9.jar:/opt/mapr/hbase/hbase-0.98.9/
bin/../lib/jersey-core-1.8.jar:/opt/mapr/hbase/hbase-0.
98.9/bin/../lib/jersey-guice-1.9.jar:/opt/mapr/hbase/hbase-

Re: hbase bulk load with map reduce error

2016-06-11 Thread Ted Yu 
Which version of hbase / Hadoop are you using ?
(So that line number matching can be more accurate)

It would be good if you can show your code snippet. 

Thanks 

> On Jun 11, 2016, at 12:57 AM, Jilani Shaik  wrote:
> 
> Hi,
> 
> I am trying to do hbase table bulk load from data file using map reduce. in 
> the process I am getting the Null pointer exception during the shuffle and 
> sort. I have both mapper and reducer in the flow. where as mapper is ending 
> with success and reducer not being invoked even setup.
> 
> It is showing the below error in terminal where the job executed.
> 
> Error: java.lang.NullPointerException
> at 
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
> at com.google.common.base.Splitter.split(Splitter.java:375)
> at 
> org.apache.hadoop.hbase.mapreduce.ImportTsv$TsvParser.(ImportTsv.java:151)
> at 
> org.apache.hadoop.hbase.mapreduce.TextSortReducer.setup(TextSortReducer.java:105)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:168)
> at 
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> 
> 
> Please let me know what could be the issue.
> 
> 
> Thanks,
> Jilani