Re: hbase bulk load with map reduce error
/../lib/jersey-json-1.8.jar:/opt/mapr/hbase/hbase- 0.98.9/bin/../lib/jersey-server-1.8.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/jets3t-0.9.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ jettison-1.3.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ jetty-6.1.26.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ jetty-sslengine-6.1.26.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/jetty-util-6.1.26.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/joni-2.1.2.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/jruby-complete-1.6.8.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/jsch-0.1.42.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ jsp-2.1-6.1.14.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ jsp-api-2.1-6.1.14.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/jsr305-1.3.9.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/junit-4.11.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ leveldbjni-all-1.8.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/libthrift-0.9.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/log4j-1.2.17.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/metrics-core-2.2.0.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/netty-3.6.6.Final.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/paranamer-2.3.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/servlet-api-2.5.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/snappy-java-1.0.4.1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/xmlenc-0.52.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/xz-1.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ zookeeper-3.4.5-mapr-1406.jar:/opt/mapr/lib/libprotodefs-4. 0.2-mapr.jar:/opt/mapr/lib/baseutils-4.0.2-mapr.jar:/opt/ mapr/lib/JPam-1.1.jar:/opt/mapr/hadoop/hadoop-2.5.1/etc/ hadoop:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/ common/lib/*:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/ common/*:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/ hdfs:/opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/hdfs/lib/*: /opt/mapr/hadoop/hadoop-2.5.1/share/hadoop/hdfs/*:/opt/mapr/ hadoop/hadoop-2.5.1/share/hadoop/yarn/lib/*:/opt/mapr/ hadoop/hadoop-2.5.1/share/hadoop/yarn/*:/opt/mapr/hadoop/hadoop-2.5.1/share/ hadoop/mapreduce/lib/*:/opt/mapr/hadoop/hadoop-2.5.1/ share/hadoop/mapreduce/* Thanks, Jilani On Sat, Jun 11, 2016 at 6:39 AM, Ted Yu <yuzhih...@gmail.com> wrote: > Which version of hbase / Hadoop are you using ? > (So that line number matching can be more accurate) > > It would be good if you can show your code snippet. > > Thanks > > On Jun 11, 2016, at 12:57 AM, Jilani Shaik <jilani2...@gmail.com> wrote: > > Hi, > > I am trying to do hbase table bulk load from data file using map reduce. > in the process I am getting the Null pointer exception during the shuffle > and sort. I have both mapper and reducer in the flow. where as mapper is > ending with success and reducer not being invoked even setup. > > It is showing the below error in terminal where the job executed. > > Error: java.lang.NullPointerException > > at > com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191) > > at com.google.common.base.Splitter.split(Splitter.java:375) > > at > org.apache.hadoop.hbase.mapreduce.ImportTsv$TsvParser.(ImportTsv.java:151) > > at > org.apache.hadoop.hbase.mapreduce.TextSortReducer.setup(TextSortReducer.java:105) > > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:168) > > at > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) > > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566) > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > > > Please let me know what could be the issue. > > > Thanks, > Jilani > >
HBase MasterNotRunningException
Hi, I have installed and configured Hadoop(hadoop-1.0.3) and HBase(hbase-0.94.1) in a single Fedora linux box, where HBase as pseudo cluster setup. I am able to connect the HBase suing shell from console and able to create and scan table. But when I tried to contact the same using java application I am not able to connect the HBase. Output of jps as follows 16872 HRegionServer 16603 HQuorumPeer 15821 DataNode 16047 JobTracker 16667 HMaster 17984 Jps 15680 NameNode 15959 SecondaryNameNode 16179 TaskTracker The error log for zoo keeper of HBase is as follows 2012-08-30 18:44:19,231 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.265.47.219:58357 2012-08-30 18:44:19,232 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.265.47.219:58357 2012-08-30 18:44:19,236 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x13977a18091000e with negotiated timeout 18 for client /192.265.47.219:58357 2012-08-30 18:45:16,245 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination for sessionid: 0x13977a18091000e 2012-08-30 18:45:16,252 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.265.47.219 :58357 which had sessionid 0x13977a18091000e 2012-08-30 18:45:16,256 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.265.47.219:58373 2012-08-30 18:45:16,257 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.265.47.219:58373 2012-08-30 18:45:16,264 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x13977a18091000f with negotiated timeout 18 for client /192.265.47.219:58373 2012-08-30 18:46:21,279 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination for sessionid: 0x13977a18091000f 2012-08-30 18:46:21,286 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.265.47.219:58373 which had sessionid 0x13977a18091000f 2012-08-30 18:46:21,291 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.265.47.219:58391 2012-08-30 18:46:21,292 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.265.47.219:58391 2012-08-30 18:46:21,298 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x13977a180910010 with negotiated timeout 18 for client /192.265.47.219:58391 2012-08-30 18:46:53,302 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination for sessionid: 0x13977a180910010 2012-08-30 18:46:53,309 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.265.47.219:58391 which had sessionid 0x13977a180910010 Below is the exception from java program Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ZooKeeper close INFO: Session: 0x13977a18091000e closed Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$EventThread run INFO: EventThread shut down Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ZooKeeper init INFO: Initiating client connection, connectString= 192.265.47.222:2181 sessionTimeout=18 watcher=hconnection Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$SendThread startConnect INFO: Opening socket connection to server / 192.265.47.222:2181 Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$SendThread primeConnection INFO: Socket connection established to 192.265.47.222/192.265.47.222:2181, initiating session Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$SendThread onConnected INFO: Session establishment complete on server 192.265.47.222/192.265.47.222:2181, sessionid = 0x13977a18091000f, negotiated timeout = 18 Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ZooKeeper close INFO: Session: 0x13977a18091000f closed Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ClientCnxn$EventThread run INFO: EventThread shut down Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ZooKeeper init INFO: Initiating client connection, connectString=192.265.47.222:2181 sessionTimeout=18 watcher=hconnection Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ClientCnxn$SendThread startConnect INFO: Opening socket connection to server /192.265.47.222:2181 Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ClientCnxn$SendThread primeConnection INFO: Socket connection established to 192.265.47.222/192.265.47.222:2181, initiating session Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ClientCnxn$SendThread onConnected INFO: Session establishment complete on server 192.265.47.222/192.265.47.222:2181, sessionid = 0x13977a180910010, negotiated timeout = 18 Aug 30, 2012 6:46:50 PM org.apache.zookeeper.ZooKeeper close INFO: Session: 0x13977a180910010 closed 12/08/30 18:46:50 org.apache.hadoop.hbase.MasterNotRunningException: Retried 10 times 12/08/30 18:46:50 at
Re: HBase MasterNotRunningException
I am able to connect to HBase using Java client if the client is on the same box where Hadoop and HBase are installed. If the client is on other box either windows or linux, I am getting the error as below org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client Please suggest me how to identify and fix the issue. Thank You, Jilani On Thu, Aug 30, 2012 at 7:00 PM, Jilani Shaik jilani2...@gmail.com wrote: Hi, I have installed and configured Hadoop(hadoop-1.0.3) and HBase(hbase-0.94.1) in a single Fedora linux box, where HBase as pseudo cluster setup. I am able to connect the HBase suing shell from console and able to create and scan table. But when I tried to contact the same using java application I am not able to connect the HBase. Output of jps as follows 16872 HRegionServer 16603 HQuorumPeer 15821 DataNode 16047 JobTracker 16667 HMaster 17984 Jps 15680 NameNode 15959 SecondaryNameNode 16179 TaskTracker The error log for zoo keeper of HBase is as follows 2012-08-30 18:44:19,231 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.265.47.219:58357 2012-08-30 18:44:19,232 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.265.47.219:58357 2012-08-30 18:44:19,236 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x13977a18091000e with negotiated timeout 18 for client /192.265.47.219:58357 2012-08-30 18:45:16,245 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination for sessionid: 0x13977a18091000e 2012-08-30 18:45:16,252 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.265.47.219 :58357 which had sessionid 0x13977a18091000e 2012-08-30 18:45:16,256 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.265.47.219:58373 2012-08-30 18:45:16,257 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.265.47.219:58373 2012-08-30 18:45:16,264 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x13977a18091000f with negotiated timeout 18 for client /192.265.47.219:58373 2012-08-30 18:46:21,279 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination for sessionid: 0x13977a18091000f 2012-08-30 18:46:21,286 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.265.47.219:58373 which had sessionid 0x13977a18091000f 2012-08-30 18:46:21,291 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.265.47.219:58391 2012-08-30 18:46:21,292 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.265.47.219:58391 2012-08-30 18:46:21,298 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x13977a180910010 with negotiated timeout 18 for client /192.265.47.219:58391 2012-08-30 18:46:53,302 INFO org.apache.zookeeper.server.PrepRequestProcessor: Processed session termination for sessionid: 0x13977a180910010 2012-08-30 18:46:53,309 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.265.47.219:58391 which had sessionid 0x13977a180910010 Below is the exception from java program Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ZooKeeper close INFO: Session: 0x13977a18091000e closed Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$EventThread run INFO: EventThread shut down Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ZooKeeper init INFO: Initiating client connection, connectString= 192.265.47.222:2181 sessionTimeout=18 watcher=hconnection Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$SendThread startConnect INFO: Opening socket connection to server / 192.265.47.222:2181 Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$SendThread primeConnection INFO: Socket connection established to 192.265.47.222/192.265.47.222:2181, initiating session Aug 30, 2012 6:45:13 PM org.apache.zookeeper.ClientCnxn$SendThread onConnected INFO: Session establishment complete on server 192.265.47.222/192.265.47.222:2181, sessionid = 0x13977a18091000f, negotiated timeout = 18 Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ZooKeeper close INFO: Session: 0x13977a18091000f closed Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ClientCnxn$EventThread run INFO: EventThread shut down Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ZooKeeper init INFO: Initiating client connection, connectString=192.265.47.222:2181 sessionTimeout=18 watcher=hconnection Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ClientCnxn$SendThread startConnect INFO: Opening socket connection to server /192.265.47.222:2181 Aug 30, 2012 6:46:18 PM org.apache.zookeeper.ClientCnxn$SendThread primeConnection INFO: Socket connection established to 192.265.47.222
Not connecting to HBase using Java from web application
Hi,
We have a linux instance for HBase. Now I am trying to connect to HBase
using Java, when I tried using a simple program it is connecting to HBase
and doing the the operations like create and get the table rows etc. But
the same code I used in my application as a ear file and deployed its not
running. I am not getting any exceptions also. Even I tried for
RuntimeException catching to know the problem. But still not getting the
exceptions.
Please help me in resolving this issue.
Please see the below code snippet which is running as a Java console
application.
Configuration conf = HBaseConfiguration.create();
//conf.set(hbase.zookeeper.quorum, 10.0.100.90); // this property
added in the hbase-site.xml
conf.addResource(new
Path(C:/jilani/workspace/hbaseproj1/src/com/demo/hbase/hbase-site.xml));
HBaseAdmin hbase = new HBaseAdmin(conf);
// will create a table
HTableDescriptor desc = new HTableDescriptor(demo1);
HColumnDescriptor meta = new HColumnDescriptor(personal.getBytes());
HColumnDescriptor prefix = new HColumnDescriptor(account.getBytes());
desc.addFamily(meta);
desc.addFamily(prefix);
hbase.createTable(desc);
hbase.close();
// will create a table
The same code I have used in Java file but it is packaged as jar under ear
for my application. The ear and app server both on the same box where the
java console application running. But its not working.
I am getting one security exception in Java console application but still
it is giving me the required outputs like connecting to HBase and getting
data from HBase table etc. Where as in ear may be it is not connecting to
the Hbase. When I used the debug mode It is coming out from the below line
of code
HBaseAdmin hbase = new HBaseAdmin(conf);
Please see the below output from my console for the same
12/08/01 17:41:11 WARN client.ZooKeeperSaslClient: SecurityException:
java.lang.SecurityException: Unable to locate a login configuration
occurred when trying to find JAAS configuration.
12/08/01 17:41:11 INFO client.ZooKeeperSaslClient: Client will not
SASL-authenticate because the default JAAS configuration section 'Client'
could not be found. If you are not using SASL, you may ignore this. On the
other hand, if you expected SASL to work, please fix your JAAS
configuration.
12/08/01 17:41:11 INFO zookeeper.ClientCnxn: Socket connection established
to systemName/10.0.100.90:2181, initiating session
12/08/01 17:41:11 INFO zookeeper.ClientCnxn: Session establishment complete
on server systemName/10.0.100.90:2181, sessionid = 0x138c9dba4c7004d,
negotiated timeout = 1
Demo.main() i: 0columnDescriptors[i] {NAME = 'account', BLOOMFILTER =
'NONE', REPLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE',
MIN_VERSIONS = '0', TTL = '2147483647', BLOCKSIZE = '65536', IN_MEMORY
= 'false', BLOCKCACHE = 'true'}
Demo.main() i: 1columnDescriptors[i] {NAME = 'personal', BLOOMFILTER =
'NONE', REPLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE',
MIN_VERSIONS = '0', TTL = '2147483647', BLOCKSIZE = '65536', IN_MEMORY
= 'false', BLOCKCACHE = 'true'}
Thank You,
Jilani
