>> Closing the HTable itself doesn't close the connections... Interesting... I must confess I didn't know this.
So in addition to closing the table is there anything else we need to do to close connection? Or is the opening/closing of connections done automatically by HBase? Please let me know. Thanks. --- On Fri, 10/9/09, Jean-Daniel Cryans <[email protected]> wrote: From: Jean-Daniel Cryans <[email protected]> Subject: Re: Test fail randomly To: [email protected] Date: Friday, October 9, 2009, 11:16 AM Julien, Closing the HTable itself doesn't close the connections, it just flushes anything that may still be in the write buffer (isn't your case). To close all connections, call HConnectionManager.deleteAllConnections(true). A client connection to Zookeeper is made by the Master, the RegionServer, the HBase clients and in version 0.20.0 by the RegionHistorian. Some of those even carry more than 1 connection so 5 connections is too low. WRT the "Exception 2" thing, I tried and got it some times. This is an expected behavior, at least it's by design. If you delete and put too fast, you may be able to have both operations at the same timestamp in which case the delete wins... so your row appears to be deleted. If you need a higher level of precision, you have to provide your own timestamps or put a sleep of some milliseconds after the deletes. Also, in your code you delete 10 times the same row. Maybe you already know that, but deleting the row key deletes the whole row with all the data you inserted so you only need to delete once. J-D On Fri, Oct 9, 2009 at 1:06 PM, Julien Ruchaud <[email protected]> wrote: > I am continue test, I have set > base.zookeeper.property.maxClientCnxns = 5 and close htable each get, > put and delete, I have got the same error. is it normal ? > > Julien Ruchaud > > Le Fri, 9 Oct 2009 17:48:37 +0200, > Julien Ruchaud <[email protected]> a écrit : > >> J-D, >> >> Yes, it is the only test. I am downloading here >> http://mir2.ovh.net/ftp.apache.org/dist/hadoop/hbase/ the 0.20.0 >> release. >> >> Julien >> >> Le Fri, 9 Oct 2009 11:47:45 -0400, >> Jean-Daniel Cryans <[email protected]> a écrit : >> >> > Julien, >> > >> > Is it the only test you are running? If so, you shouldn't have to >> > close after each operation but only at the end of the test. Are you >> > sure this is the official 0.20.0 release? >> > >> > J-D >> > >> > On Fri, Oct 9, 2009 at 11:35 AM, Julien Ruchaud >> > <[email protected]> wrote: >> > > Hi, >> > > >> > > If hbase.zookeeper.property.maxClientCnxns increase to 300, the >> > > test pass. So I test the second solution to close connection each >> > > put, get and delete, the test pass too. How do you know when to >> > > close connection ? each get, delete or put ? is it better to >> > > instanciate a new HTable each time ? >> > > >> > > Thanks >> > > >> > > Julien Ruchaud >> > > >> > > Le Fri, 9 Oct 2009 11:07:55 -0400, >> > > Jean-Daniel Cryans <[email protected]> a écrit : >> > > >> > >> Julien, >> > >> >> > >> This is a Zookeeper error that can be related to the number of >> > >> connections a ZK server can handle. To verify, set >> > >> hbase.zookeeper.property.maxClientCnxns to something a lot bigger >> > >> than the default 30 and restart HBase. >> > >> >> > >> This could happen because you run a lot of unit tests >> > >> sequentially without closing the connections. >> > >> >> > >> J-D >> > >> >> > >> On Fri, Oct 9, 2009 at 10:53 AM, Julien Ruchaud >> > >> <[email protected]> wrote: >> > >> > Hi, >> > >> > >> > >> > I am a newbie on hbase. Currently I am testing hbase 0.20.0, >> > >> > but I don't understand why my test don't run correctly >> > >> > sometimes : >> > >> > >> > >> > public void testLoop() throws Exception { >> > >> > HTable hTable = new HTable(new HBaseConfiguration(), >> > >> > Bytes.toBytes("Table")); byte[] rowId = Bytes.toBytes("Test"); >> > >> > >> > >> > for (int j = 0; j < 10; j++) { >> > >> > Put put = new Put(rowId); >> > >> > put.add(Bytes.toBytes("data"), Bytes.toBytes("1:" + >> > >> > j), Bytes.toBytes(j)); hTable.put(put); >> > >> > } >> > >> > >> > >> > for (int j = 0; j < 10; j++) { >> > >> > Get get = new Get(rowId); >> > >> > Result row = hTable.get(get); >> > >> > if(row.getValue(Bytes.toBytes("data"), >> > >> > Bytes.toBytes("1:" + j)) == null) { throw new Exception("Error >> > >> > 1"); } >> > >> > } >> > >> > >> > >> > for (int j = 0; j < 10; j++) { >> > >> > Delete delete = new Delete(rowId); >> > >> > hTable.delete(delete); >> > >> > } >> > >> > >> > >> > for (int j = 0; j < 10; j++) { >> > >> > Put put = new Put(rowId); >> > >> > put.add(Bytes.toBytes("data"), Bytes.toBytes("2:" + >> > >> > j), Bytes.toBytes(j)); hTable.put(put); >> > >> > } >> > >> > >> > >> > for (int j = 0; j < 10; j++) { >> > >> > Get get = new Get(rowId); >> > >> > Result row = hTable.get(get); >> > >> > if(row.getValue(Bytes.toBytes("data"), >> > >> > Bytes.toBytes("2:" + j)) == null) { throw new Exception("Error >> > >> > 2"); } >> > >> > } >> > >> > } >> > >> > >> > >> > In fact I have got exception "Error 2" with a exception in >> > >> > hbase log : >> > >> > >> > >> > 2009-10-09 16:39:34,829 WARN >> > >> > org.apache.zookeeper.server.NIOServerCnxn: Exception causing >> > >> > close of session 0x124393fc5390046 due to java.io.IOException: >> > >> > Read error 2009-10-09 16:39:34,829 DEBUG >> > >> > org.apache.zookeeper.server.NIOServerCnxn: IOException stack >> > >> > trace java.io.IOException: Read error at >> > >> > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:366) >> > >> > at >> > >> > org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:220) >> > >> > 2009-10-09 16:39:34,830 INFO >> > >> > org.apache.zookeeper.server.NIOServerCnxn: closing >> > >> > session:0x124393fc5390046 NIOServerCnxn: >> > >> > java.nio.channels.SocketChannel[connected local=/127.0.0.1:2181 >> > >> > remote=/127.0.0.1:48870] 2009-10-09 16:39:34,830 DEBUG >> > >> > org.apache.zookeeper.server.NIOServerCnxn: ignoring exception >> > >> > during input shutdown java.net.SocketException: Transport >> > >> > endpoint is not connected at >> > >> > sun.nio.ch.SocketChannelImpl.shutdown(Native Method) at >> > >> > sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640) >> > >> > at >> > >> > sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360) >> > >> > at >> > >> > org.apache.zookeeper.server.NIOServerCnxn.close(NIOServerCnxn.java:841) >> > >> > at >> > >> > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:492) >> > >> > at >> > >> > org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:220) >> > >> > >> > >> > Thanks in advance for your input >> > >> > >> > >> > Julien Ruchaud >> > >> > >> > > >
