Hi - As J-D says, the original error shows that the client is trying to connect to localhost instead of the distributed ZK ensemble, which should be configured in hbase.zookeeper.quorum.
The second set of errors are exactly as they say - a create operation is failing because the node to be created already exists. It is a bit strange if the errors occur when you're just doing an list command from the shell - I don't think they silently insert any create operations. Can you just make sure the errors occur exactly when you issue a list command? There's nothing to worry about, at least in terms of ZK stability, from the NodeExistsException - it's an 'expected' exception, if that's not a contradiction in terms, and is simply caught and signalled back to the client. cheers, Henry On 26 February 2010 14:15, Michael Segel <michael_se...@hotmail.com> wrote: > > Setting up a development cluster. > > Using Cloudera's latest release which has HBase-20.3. > We have 3 nodes running ZooKeeper which is managed by HBase. > We have a quorum set up. > > One of the developers ran in to this error: > 10/02/26 07:31:48 WARN zookeeper.ClientCnxn: Exception closing > session 0x0 to > sun.nio.ch.selectionkeyi...@6708f8e0java.net.connectexception: Connection > refused at > sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:933)10/02/26 > 07:31:48 WARN zookeeper.ClientCnxn: Ignoring exception > during shutdown inputjava.nio.channels.ClosedChannelException at > sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638) > at > sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360) at > org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)10/02/26 > 07:31:48 WARN zookeeper.ClientCnxn: Ignoring exception > during shutdown outputjava.nio.channels.ClosedChannelException at > sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:649) > at > sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368) at > org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004) > at > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)10/02/26 > 07:31:49 WARN zookeeper.ZooKeeperWrapper: Failed to create > /hbase -- check quorum servers, > currently=localhost:2181org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for /hbase > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [SNIP] > The strange thing is that we have a quorum but the error says > currently=localhost. > Checking the logs, I found that I had an error ... KeeperErrorCode = > NodeExistsI set the logging to DEBUG and restarted both HBase and Hadoop. > (When both down checked that no zombie processes were left running.) > Now I get the following block in the zookeeper log:2010-02-26 14:35:36,435 > WARN org.apache.zookeeper.server.PrepRequestProcessor: Got exception when > processing sessionid:0x270bfd523f0003 type:create cxid:0x2 > zxid:0xfffffffffffffffe txntype:unknown n/a > org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = > NodeExists > at > org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:245) > at > org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114) > 2010-02-26 14:35:36,435 DEBUG > org.apache.zookeeper.server.quorum.CommitProcessor: Processing request:: > sessionid:0x270bfd523f0003 type:create cxid:0x2 zxid:0x6300000019 txntype:-1 > n/a > 2010-02-26 14:35:36,436 DEBUG org.apache.zookeeper.server.quorum.Leader: > Proposing:: sessionid:0x270bfd523f0003 type:create cxid:0x2 > zxid:0x6300000019 txntype:-1 n/a > 2010-02-26 14:35:36,439 DEBUG org.apache.zookeeper.server.quorum.Leader: > Count for zxid: 0x6300000019 is 1 > 2010-02-26 14:35:36,445 DEBUG org.apache.zookeeper.server.quorum.Leader: > Count for zxid: 0x6300000019 is 2 > 2010-02-26 14:35:36,445 DEBUG > org.apache.zookeeper.server.quorum.CommitProcessor: Committing request:: > sessionid:0x270bfd523f0003 type:create cxid:0x2 zxid:0x6300000019 txntype:-1 > n/a > 2010-02-26 14:35:36,445 DEBUG org.apache.zookeeper.server.quorum.Leader: > outstanding is 0 > 2010-02-26 14:35:36,445 DEBUG > org.apache.zookeeper.server.FinalRequestProcessor: Processing request:: > sessionid:0x270bfd523f0003 type:create cxid:0x2 zxid:0x6300000019 txntype:-1 > n/a > > Now this happens when I run a command in the hbase shell. (Like list, > status, ...) > I'd like to understand what is happening and what I can do to fix this? > I'm not sure why the user had Zookeeper errors showing local host but the > larger concern is what is happening to cause this? > Earlier I had i/o errors mentioning /hbase/root but they went away when I > took everything down.Since this is still a development machine, I may just > blow away /hbase in HDFS and see if I can come up clean. > TIA. > -Mike > > > _________________________________________________________________ > Hotmail: Free, trusted and rich email service. > http://clk.atdmt.com/GBL/go/201469228/direct/01/ > -- Henry Robinson Software Engineer Cloudera 415-994-6679
