I'd look in the zookeeper logs for more clues. We saw something similar to this, and found OutOfMemoryErrors in the ZK logs.
Mark Vigeant wrote: > So I said I'd tell you if the problem came up again, so here's the error that > gets thrown at me over and over again: > > 09/10/02 23:59:23 WARN zookeeper.ClientCnxn: Exception closing session 0x0 to > sun.nio.ch.selectionkeyi...@92eb76 > java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 lim=4 > cap=4] > at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:653) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:897) > 09/10/02 23:59:23 WARN zookeeper.ClientCnxn: Ignoring exception during > shutdown input > java.net.SocketException: Transport endpoint is not connected > at sun.nio.ch.SocketChannelImpl.shutdown(Native Method) > at > sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:951) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:922 > > This is actually different from what it was the other day. Any clue what's > going on here? > > -----Original Message----- > From: [email protected] [mailto:[email protected]] On Behalf Of Jean-Daniel > Cryans > Sent: Tuesday, October 06, 2009 11:54 AM > To: [email protected] > Subject: Re: NoNode Exception? > > Care to show us your configuration? Are you managing the zookeeper > ensemble yourself or do you let HBase do it? > > Can I see a full stack trace along with some lines before and after? > Try to get the first one, when things go awry. > > J-D > > On Tue, Oct 6, 2009 at 11:49 AM, Mark Vigeant > <[email protected]> wrote: > >> Yeah I made a data directory for zookeeper, so that can't be it. >> >> And in the logs all I see are NoNodeExceptions. I reconfigured it so that >> zookeeper was no longer in standalone mode but the error is still occurring. >> Perhaps I have to reformat everything then configure it in distributed mode? >> >> -----Original Message----- >> From: [email protected] [mailto:[email protected]] On Behalf Of >> Jean-Daniel Cryans >> Sent: Tuesday, October 06, 2009 11:06 AM >> To: [email protected] >> Subject: Re: NoNode Exception? >> >> Mark, >> >> By default the data is stored in /tmp which is cleared regularly, did >> you change hbase.tmp.dir or hbase.zookeeper.property.dataDir? >> >> Also do you get other exceptions in the master and regionserver logs? >> >> J-D >> >> On Tue, Oct 6, 2009 at 10:57 AM, Mark Vigeant >> <[email protected]> wrote: >> >>> Hey- >>> >>> So I've been messing around with HBase 0.20.0 for the past couple of weeks >>> on some virtual machines (with ubuntu), and for whatever reason every week >>> or so I get thrown a ton of exceptions and have no idea what to do except >>> format the nodes and reinstall everything. Basically I get this : >>> >>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = >>> NoNode for /hbase/master >>> >>> but it's silly because I know the master is running (when typing jps it >>> says HMaster and HQuorumPeer are running fine) so I was wondering if there >>> was a patch or something that fixes this... I don't think this is because >>> of something I did because it occurs randomly. I don't know. I throw myself >>> at the mercy of the listserve. >>> >>> Mark Vigeant >>> RiskMetrics Group, Inc. >>> >>> >>> -- Eric Bowman Boboco Ltd [email protected] http://www.boboco.ie/ebowman/pubkey.pgp +35318394189/+353872801532
