Yes, I do have that set. In my case, the quorum is just a single machine, that is also the Hbase master. And I'm pretty sure that the client is reaching it, because as soon as I start my client, I see a bunch of the "NodeExists" exceptions in the zookeeper log.
-Adam On 1/13/10 1:07 PM, "Jean-Daniel Cryans" <[email protected]> wrote: > Oh, so HBase is working fine but you have a remote client connectivity > problem. You gave the good hbase.zookeeper.quorum to HTable? > > J-D > > On Wed, Jan 13, 2010 at 1:04 PM, Adam Silberstein > <[email protected]> wrote: >> No, nothing at the master, other than a bunch of NodeExists exceptions in >> the zookeeper log. >> >> Something else I've found is that I can add records to the table through the >> shell, without problems. It's only from the java client, on a separate box, >> that I'm having problems. >> >> Thanks, >> Adam >> >> >> On 1/13/10 12:08 PM, "Jean-Daniel Cryans" <[email protected]> wrote: >> >>> It means the root region likely wasn't assigned, anything in the >>> master about it? >>> >>> J-D >>> >>> On Wed, Jan 13, 2010 at 11:43 AM, Adam Silberstein >>> <[email protected]> wrote: >>>> I have version 0.20.2. The client requests hang. >>>> And, I looked in the region server logs...here is the likely culprit: >>>> >>>> java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: >>>> KeeperErrorCode = NoNode for /hbase/root-region-server >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>> e >>>> perWrapper.java:332) >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWra>>>> p >>>> per.java:318) >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Z>>>> o >>>> oKeeperWrapper.java:231) >>>> at >>>> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:4>>>> 4 >>>> 2) >>>> at java.lang.Thread.run(Thread.java:619) >>>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: >>>> KeeperErrorCode = NoNode for /hbase/root-region-server >>>> at >>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:102) >>>> at >>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42) >>>> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892) >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>> e >>>> perWrapper.java:328) >>>> ... 4 more >>>> java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException: >>>> KeeperErrorCode = NoNode for /hbase/root-region-server >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>> e >>>> perWrapper.java:332) >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWra>>>> p >>>> per.java:318) >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readRootRegionLocation(Z>>>> o >>>> oKeeperWrapper.java:231) >>>> at >>>> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:4>>>> 4 >>>> 2) >>>> at java.lang.Thread.run(Thread.java:619) >>>> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: >>>> KeeperErrorCode = NoNode for /hbase/root-region-server >>>> at >>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:102) >>>> at >>>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42) >>>> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:892) >>>> at >>>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKe>>>> e >>>> perWrapper.java:328) >>>> ... 4 more >>>> >>>> Is there zk state on the region servers? Maybe when wiping the system, I >>>> should have wiped something on them. >>>> >>>> Thanks, >>>> Adam >>>> >>>> On 1/13/10 11:33 AM, "Jean-Daniel Cryans" <[email protected]> wrote: >>>> >>>>> Those node exists are usually harmless, I would like the see what's on >>>>> the HBase region server and master side. >>>>> >>>>> You say it cannot do any operation but you don't see exceptions >>>>> client-side? It just hangs? Which version is it? >>>>> >>>>> Thx, >>>>> >>>>> J-D >>>>> >>>>> On Wed, Jan 13, 2010 at 11:25 AM, Adam Silberstein >>>>> <[email protected]> wrote: >>>>>> Hi, >>>>>> I set up a new Hbase cluster and am seeing an error I haven't seen >>>>>> before. >>>>>> When I start up Hbase I have this NodeExists error in the zookeeper log: >>>>>> >>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode >>>>>> = >>>>>> NodeExists >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>> >> >> >> s >>>>>> or.java:245) >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>> >> >> >> a >>>>>> va:114) >>>>>> 2010-01-13 11:10:47,503 WARN >>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when >>>>>> processing sessionid:0x1262918396b0002 type:create cxid:0xc >>>>>> zxid:0xfffffffffffffffe txntype:unknown n/a >>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode >>>>>> = >>>>>> NodeExists >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>> >> >> >> s >>>>>> or.java:245) >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>> >> >> >> a >>>>>> va:114) >>>>>> >>>>>> I've read that may be harmless. But then when I run the java client, it >>>>>> cannot do any operations. And I see a similar NodeExists error in the >>>>>> zookeeper log. >>>>>> >>>>>> >>>>>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn: >>>>>> Connected to /203.104.21.210:36928 lastZxid 0 >>>>>> 2010-01-13 11:11:24,661 INFO org.apache.zookeeper.server.NIOServerCnxn: >>>>>> Creating new session 0x1262918396b0004 >>>>>> 2010-01-13 11:11:24,664 INFO org.apache.zookeeper.server.NIOServerCnxn: >>>>>> Finished init of 0x1262918396b0004 valid:true >>>>>> 2010-01-13 11:11:24,973 WARN >>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when >>>>>> processing sessionid:0x1262918396b0004 type:create cxid:0x2 >>>>>> zxid:0xfffffffffffffffe txntype:unknown n/a >>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode >>>>>> = >>>>>> NodeExists >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>> >> >> >> s >>>>>> or.java:245) >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>> >> >> >> a >>>>>> va:114) >>>>>> 2010-01-13 11:11:33,088 WARN org.apache.zookeeper.server.NIOServerCnxn: >>>>>> Exception causing close of session 0x1262918396b0004 due to >>>>>> java.io.IOException: Read error >>>>>> 2010-01-13 11:11:33,089 INFO org.apache.zookeeper.server.NIOServerCnxn: >>>>>> closing session:0x1262918396b0004 NIOServerCnxn: >>>>>> java.nio.channels.SocketChannel[connected local=/203.104.21.210:2181 >>>>>> remote=/203.104.21.210:36928] >>>>>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn: >>>>>> Connected to /203.104.21.212:54837 lastZxid 0 >>>>>> 2010-01-13 11:11:42,709 INFO org.apache.zookeeper.server.NIOServerCnxn: >>>>>> Creating new session 0x1262918396b0005 >>>>>> 2010-01-13 11:11:42,711 INFO org.apache.zookeeper.server.NIOServerCnxn: >>>>>> Finished init of 0x1262918396b0005 valid:true >>>>>> 2010-01-13 11:11:42,711 WARN >>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Got exception when >>>>>> processing sessionid:0x1262918396b0005 type:create cxid:0x1 >>>>>> zxid:0xfffffffffffffffe txntype:unknown n/a >>>>>> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode >>>>>> = >>>>>> NodeExists >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProces>> >> >> >> s >>>>>> or.java:245) >>>>>> at >>>>>> >> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.j>> >> >> >> a >>>>>> va:114) >>>>>> 2010-01-13 11:12:06,002 INFO >>>>>> org.apache.zookeeper.server.SessionTrackerImpl: >>>>>> Expiring session 0x1262918396b0004 >>>>>> 2010-01-13 11:12:06,002 INFO org.apache.zookeeper.server.ZooKeeperServer: >>>>>> Expiring session 0x1262918396b0004 >>>>>> 2010-01-13 11:12:06,002 INFO >>>>>> org.apache.zookeeper.server.PrepRequestProcessor: Processed session >>>>>> termination request for id: 0x1262918396b0004 >>>>>> >>>>>> I've tried wiping the whole setup, including the zookeeper files that >>>>>> Hbase >>>>>> puts in /tmp and formatting HDFS. But I get the same problem again. I'm >>>>>> not seeing any exceptions on the client-side. >>>>>> >>>>>> Thanks, >>>>>> Adam >>>>>> >>>>>> >>>> >>>> >> >>
