Hello,
I have an issue regarding storing data in HBase table using
pig. The details are as follows.
I have hadoop-2.5.1 cluster with one master and one data node, each having
2GB RAM. Upon this in the master I have HBase-1.1.4 in pseudo distributed
mode. Till this everything is working fine. Then I have installed pig.
First I untarred pig-15 src-tar .gz and then untarred the pig-15.tar.gz.
Then I executed the command i.e.
ant clean tar -Dhadoopversion=23 -Dhbase95.version=1.1.2
-Dforrest.home=<downloaded forrest location>
It worked succesfully. Next I went to grunt shell in mapreduce mode. I am
able to load an input file from hdfs into a pig variable using load
command. My input file contains only 2 records.
1975,32
1950,45
I am also successfully been able to dump the data. But when I try to store
the dumped data into hbase table , I see the following
*2016-05-30 10:28:47,461 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete2016-05-30 10:28:47,462 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Running jobs are [job_1464584017709_0003]2016-05-30 10:29:22,722 [main]
INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 50% complete2016-05-30 10:29:22,722 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Running jobs are [job_1464584017709_0003]2016-05-30 11:11:30,888 [main]
WARN
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to
stop immediately on failure.2016-05-30 11:11:30,889 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- job job_1464584017709_0003 has failed! Stop running all dependent
jobs2016-05-30 11:11:30,889 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete2016-05-30 11:11:30,964 [main] INFO
org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at
hadoop-master/192.168.36.10:8032 <http://192.168.36.10:8032>2016-05-30
11:11:31,084 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate -
Application state is completed. FinalApplicationStatus=FAILED. Redirecting
to job history server2016-05-30 11:11:32,227 [main] INFO
org.apache.hadoop.mapred.ClientServiceDelegate - Application state is
completed. FinalApplicationStatus=FAILED. Redirecting to job history
server2016-05-30 11:11:32,614 [main] INFO
org.apache.hadoop.mapred.ClientServiceDelegate - Application state is
completed. FinalApplicationStatus=FAILED. Redirecting to job history
server2016-05-30 11:11:32,741 [main] INFO
org.apache.hadoop.mapred.ClientServiceDelegate - Application state is
completed. FinalApplicationStatus=FAILED. Redirecting to job history
server2016-05-30 11:11:33,240 [main] ERROR
org.apache.pig.tools.pigstats.PigStats - ERROR 0: java.io.IOException:
org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException):
java.lang.NullPointerException at
org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getTaskReports(HistoryClientService.java:319)
at
org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getTaskReports(MRClientProtocolPBServiceImpl.java:185)
at
org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:285)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928) at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013) at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:415) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)2016-05-30
11:11:33,240 [main] ERROR
org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce
job(s) failed!2016-05-30 11:11:33,240 [main] INFO
org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt
Features2.5.1 0.15.0-SNAPSHOT hduser 2016-05-30 10:28:44
2016-05-30 11:11:33 UNKNOWNFailed!Failed Jobs:JobId Alias
Feature Message Outputsjob_1464584017709_0003 rit MAP_ONLY
Message: Job failed! hbase://T1,Input(s):Failed to read data from
"/input/god.txt"Output(s):Failed to produce result in
"hbase://T1"Counters:Total records written : 0Total bytes written :
0Spillable Memory Manager spill count : 0Total bags proactively spilled:
0Total records proactively spilled: 0Job
DAG:job_1464584017709_00032016-05-30 11:11:33,241 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed!grunt>*
For your information the store command is taking aprox 1 hour to get the
error above. Kindly help as I am very new to this environment and is not
very comfortable with java.
Thanks and Regards,
Rituparna Sinha,
Asst. Professor,
Dept. of IT
Heritage Institute of Technology
Thanks and Regards,
Rituparna Sinha,
Asst. Professor,
Dept. of IT
Heritage Institute of Technology
--
*Disclaimer:*
This e-mail contains confidential information belonging to Heritage
Institute of Technology & is intended solely for the Addresses. The
unauthorized disclosure, use dissemination or copying (either whole or
partial of this e-mail, or any information it contains), is prohibited.
E-mail is susceptible to alteration and their integrity cannot be
guaranteed. Heritage Institute of Technology shall not be liable of this
e-mail if modified or falsified. If you are not the intended recipient of
this e-mail, please delete it immediately from your system & notify the
sender of the wrong delivery and the mail deletion.
